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9.0  BRIEF  OUTLINE  OF  RESEARCH  FINDINGS 
9.1  Overview  of  Accomplishments 

In  late  1986,  when  the  Center  for  Intelligent  Control  Systems  (CICS)  was  formed,  we  set  out  to  investigate 
a  core  of  subjects  that  might  constitute  the  foundatimis  of  intelligent  systems.  We  recognized  that  it  would 
be  essential  to  go  beyond  the  standard  paradigms  oS  control  theory  and  signal  processing  which  deal  with 
well-defined  models  of  systems  (using  difference  or  differential  equations),  and  that  it  would  be  necessary  to 
investigate  new  models  of  uncertainty  (going  beyond  stochastic  processes)  so  as  to  be  able  to  cope  with 
uncertainty  at  difEsrent  levels  of  granularity.  We  also  recognized  that  the  new  constructs  would  need  to  deal 
with  the  structural  rqiresentatiai  of  models  at  different  levels  of  abstraction  and  that  these  representations 
should  he4>  facilitate  the  mathematical  codification  of  the  concepts  of  learning,  adaptation  and  organization. 

In  this  section  we  describe  some  of  our  research  and  educational  accomplishments. 

Research  on  Perceptual  Systems 

Understanding  natural  perceptual  systems  such  as  the  human  visual  and  auditory  systems,  and  the  synthesis 
of  artificial  perceptual  systems,  are  important  problems  in  the  study  of  intelligent  systems.  Of  course 
understanding  nanitally  occurring  percqrtual  systems  has  proven  to  be  more  difficult  than  anyone  expected 
SO  years  ago,  and  it  now  seems  that  it  will  require  inputs  from  such  diverse  fields  as  brain  and  cognitive 
science,  linguistics,  and  psychology,  as  well  as  mathematics  (geometry,  partial  differential  equations, 
random  fields),  systems,  and  computer  science. 

One  possible  view  of  the  visual  cortex  is  a  computational  one:  it  can  be  modeled  as  an  information 
processing  system  consisting  of  several  modules  apetssiag  at  different  levels.  The  goal  of  the  first  stqr  in 
vision  (early  vision)  is  to  solve  problems  such  as: 

(i)  segmenting  an  image  into  its  natural  regions 

(ii)  finding  boundaries  and  detecting  objects 

(iii)  organizing  local  infmmation  into  a  global  structural  description. 
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The  integration  of  these  computations  is  a  challenging  problem.  Mcneover,  sensed  images  will  invariably 
contain  noise  and  hence  these  tasks  will  have  to  be  perfumed  in  a  n^ust  way.  Higher  level  processes  may 
include: 


(iv)  the  use  of  structural  ideas  to  find  more  robust  texture  features 

(v)  the  use  of  effective  combinatorial  descripdons  of  shape  suitable  for  recognition  in  the  presence  of 
nmlinear  distortions. 

As  a  result  of  our  research  over  the  last  few  years,  we  now  know  how  to  attack  some  of  these  problems 
using  ideas  from  the  calculus  of  variations,  geometric  measure  theory,  and  Maritov  random  fields.  This  has 
lead  to  a  deeper  understanding  of  these  problems. 

Increasingly,  however,  we  recognize  that  a  separation  into  early  vision  and  higher  level  problems  may  not 
provide  the  greatest  insight  We  believe  that  a  central  problem  in  vision  is  that  of  representing  data  at 
different  levels  (scales)  and,  indeed,  that  recognition  is  nothing  else  but  representation  at  the  correct  level 
These  ideas  are  being  tested  from  a  variety  of  viewpoints.  For  example,  we  are  using  probabilistic  ideas 
which  take  geometry  into  account  so  as  to  understand  how  one  synthesizes  an  ideal  image  from  noisy  data 

The  line  between  perception  and  understanding  is  not  at  all  sharp.  We  know  from  Kanisza's  work  that  the 
idea  of  perception  is  extremely  complex.  A  computational  theory  which  explains  our  perception  of 
Kanisza's  illusory  flgures  is  itself  a  challenging  problem,  but  preliminary  studies  suggest  that  mathematics 
can  be  useful. 

On  a  more  pragmatic  level,  the  problem  of  aided  or  autmnatic  target  recognition  (ATR)  is  ubiquitous  and 
still  not  adequately  solved.  Continuing  progress  on  the  ATR  problem  relies  on  new  advances  in 
mathematical  iqtpioaches  to  image  analysis,  object  representation  and  recognition,  and  decision  theory. 

A  promising  attack  on  the  high-level  problem  of  recognition  of  shapes  has  been  launched.  It  is  based  on: 

G)  creating  defotmable  templates  to  represent  idealized  objects. 

(u)  developing  a  methodology  (drawn  from  statistics  and  sequential  decision  theory)  for  the  recognition 
of  rigid  objects  which  may  present  themselves  in  a  variety  of  aspects  and  fmms  due  to  rotations, 
partial  obscuration,  or  sensor  noise. 

Surprisingly,  some  reasoiuibly  advanced  mathematical  ideas  involving  infinite-dimensional  stochastic 
differential  equations  and  infinite-dimensional  manifolds  have  played  a  role  here. 

Research  on  percqmial  systems  at  the  CICS  has  been  led  by  Professon  Brockett,  Ceman,  Gidas,  Grenander, 
Ktoagos,  McClure,  Mitter,  Mumford,  Willsky,  and  YuiBe.  Details  of  their  work  may  be  found  in  their 
individual  reports  in  Section  9.2. 

Mnltlscalc  Signal  and  Image  Analysis 

The  investigation  of  multiscale  representations  of  signals  and  the  develqpment  of  multiscale  algorithms  is  a 
topic  of  considerable  current  interest  One  of  the  mote  recent  areas  of  investigation  in  multi-scale  aiudysis 
resulted  in  the  development  of  a  theory  of  multiscale  signal  representations  and  wavelet  trmisforms,  and  it 
has  frequently  been  suggested  that  it  should  be  possible  to  develop  efficient  optimal  processing  algorithms 
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based  on  these  iqjfesentadons.  The  development  of  such  algorithms  requires  the  development  of  a 
corresponding  themy  of  stochastic  processes  and  their  estimation.  Work  in  this  area  has  had  this  as  its 
objective. 

We  have  taken  some  initial  steps  towards  developing  a  theory  for  muldresolution  stochastic  processes,  and 
a  cmesponding  signal  processing  and  estimation  methodology.  The  wavelet  transform  leads  namrally  to 
the  investigation  of  stochastic  processes  on  dyadic  trees  and  a  more  general  class  of  weighted  lattices  in 
which  different  depths  in  the  tree  or  lattice  correspond  to  different  scales,  thereby  exposing  the  "time-like" 
natnre  of  the  scale  index  and  allowing  us  to  begin  the  development  of  scale-to-scale  recursive  specification 
of  multiscale  stochastic  processes.  In  particular,  in  our  initial  work  we  have  focused  on  inocesses  on  dyadic 
trees.  We  have  also  introduced  a  class  of  linear  state  space  models  evolving  "causally  in  scale"  (i.e.  from 
coarse  to  fine)  on  the  dyadic  tree.  We  have  shown  how  the  statistics  of  such  a  process  can  be  evaluated 
efficiently,  and  we  also  develop  a  generalization  of  Kalman  Filtering  and  smoothing  for  such  models. 

These  algorithms  have  some  interesting  twists.  For  example,  the  Kalman  ffne-to-coarse  recursion  involves 
not  only  the  prediction  and  measurement  update  steps  familiar  from  standard  Kalman  Filtering  but  also  a 
new  fuaon  step  arising  in  the  estimation  of  a  coarse-level  signal  description  based  on  estimates  at  fine 
levels.  This  leads  to  a  new  class  of  multiscale  Riccati  equations  whose  analysis  requires  a  corresponding 
system  theory.  We  have  provided  some  initial  steps  in  developing  this  system  theory  and  in  performing 
analysis  of  the  Riccati  equations.  We  also  have  introduced  two  other  algorithmic  siruaures.  One  involves 
the  use  of  the  Haar  transform  to  transform  the  estimation  problem  into  a  set  of  decoupled  (and  much 
simpler)  problems  at  different  scales.  The  second  algorithmic  structure  is  iterative  in  nature  and,  in  fact,  has 
exactly  the  structure  of  popular  and  highly  efficient  multigrid  aigtxithms. 

Another  part  of  our  research  on  multiscale  processes  on  dyadic  trees  has  had  as  its  aim  the  development  of  a 
theory  for  modeling  what  might  be  described  as  multiscale  autoregressive  (AR)  inocesses  ~  i.e.  AR  models 
evolving  in  scale.  In  this  work  we  develop  a  generalization  of  lattice  stmcmres  and  the  celebrated  Levinson 
and  Schur  recursion  of  time  series  analysis.  Again,  thanks  to  the  tree  strocuire  of  the  index  set,  we  find  that 
there  ate  significant  and  extremely  important  differetKes  between  our  theory  and  the  standard  time  series 
case.  In  particular  our  lattice  structures  involve  prediction  error  vecttns  that  increase  in  dimension  as  the 
order  of  the  structure  increases.  This  is  due  to  the  fact  that  the  number  of  points  on  the  tree  that  are  a 
specified  distance  or  "lag"  away  from  a  given  point  increases  as  the  lag  increases.  Also,  the  reflection 
coefficients  defining  such  a  model  satisfy  different  recursions  and  are  subjea  to  different  constraints,  due  to 
the  fact  that  there  are  more  stringent  constraints  on  a  correlation  sequence  for  a  multiscale  process  on  a  tree 
than  on  such  a  sequence  for  an  ordinary  stationary  time  series.  From  our  analysis  it  appears  that  multiscale 
AR  models  may  be  as  useful  a  model  class  of  multiscale  signal  inocessing  as  standard  AR  models  are  for  so 
many  problems  in  mott  classical  signal  processing  contexts. 

Research  on  mnltiscale  signal  and  image  analysis  at  the  CICS  has  been  led  by  Professru^  Willsky.  Details  of 
this  work  may  be  found  in  his  individual  report  in  Sectkm  9.2. 

iBtcIliient  Control 

Over  the  last  few  decades,  developments  in  control  themy  have  established  a  sound  foundation  (or  designing 
linear  control  systems,  and  a  suitable  language  for  specifying  performance  requirements  in  the  areas  of 
stochastic  control,  system  identification  and  even  stabilization  of  unknown,  but  linear,  systems.  Aspects  of 
this  theory  have  been  extended  so  as  to  include  important  classes  of  smooth  nonlinear  systems.  These 
developments  do  not,  however,  directly  address  a  number  (tf  the  most  important  issues  one  faces  in 
designing  or  analyzing  systems  which  ate  intelligent  in  the  sense  that  they  can  exhibit  the  flexible 
stmegiesandfer  opportunistic  behavior  requited  for  dealing  with  large  grained  uncertainty.  Inthefieldof 
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motion  control  -  e.g.  autonomous  vehicles,  robotics,  and  smait  machine  tools,  as  well  as  modem  chemical 
process  control  and  some  types  of  manufacturing  systems  -•  traditional  methods  must  be  augmented  with 
techniques  that  allow  the  system  to  exhibit  a  great  capacity  for  learning  about  its  environment,  making  a 
strategical  analysis,  and  reacting  sq;)propriately.  It  seems  likely  that  recent  developments  in  statistics  and  the 
continuing  work  on  probabilistic  approaches  to  learning  will  have  substantial  impact  on  the  resolution  of 
these  questions. 

Instead  of  describing  the  input  to  such  a  system  in  terms  of  a  vector  of  set  points,  or  in  terms  of  a  class  of 
trajectories  to  be  tracked,  it  is  often  mne  suitable  to  think  of  the  inputs  as  being  a  symbol  string  describing 
a  sequence  of  moves  or  strategies  that  the  system  is  to  execute.  In  this  sense  systems  of  interest  in 
intelligent  control  ate  sometimes  described  as  symbol-to-signal  transducers.  Although  the  techniques  for 
describing  the  structure  and  specifying  the  performance  of  such  systems  are  still  under  development,  tools 
for  relating  automata-like  mo^  and  differential  equation  models  are  now  emoging  from  developmoits  in 
nonlinear  system  theory.  These  dynamical  systems  methods  give  a  unified  framew(»k  for  studying 
interesting  broad  classes  of  intelligent  systems. 

By  their  very  nature,  intelligent  systems  often  consist  of  interconnected  modules  distributed  in  space. 
Because  the  feedback  control  of  interconnected  systems  requires  coordination,  it  is  necessary  for  the  system 
to  make  efficient  use  of  the  available  coirununication  resources,  and  this  includes  deciding  what  information 
is  useful  enough  to  share.  Moreover,  because  feedback  control  of  a  localized  system  may  involve  complex 
decision  making  (e.g.  path  planning)  in  order  to  facilitate  the  decision  making  processes,  it  is  often 
necessary  to  abstract  from  the  totality  the  data  available  in  order  to  reduce  the  description  of  the  problem  to 
manageable  proportions.  In  both  cases  one  is  faced  with  feedback  lo(^  in  which  substantial  data  reduction 
is  essential.  If  this  reduction  can  be  described  in  terms  finding  an  approximation  relative  to  some  metric, 
then  we  may  think  of  it  as  quantization.  On  the  other  hand,  if  the  situation  is  such  that  "high  level” 
abstractions  ate  involved,  then  new  methods,  possibly  derived  from  a  combination  of  control  and 
information  theory  will  be  required.  Such  methods  ate  now  being  explixed  by  workers  in  natural  percqrtion 
using  mathematical  techniques  such  as  template  matching.  The  application  of  these  ideas  to  real-time 
control  problems  seems  qrpropriate. 

A  broad  range  of  applications  ate  built  on  the  foundation  of  the  basic  research  in  intelligent  systems. 
Indeed,  the  concept  of  a  machine  that  can  process  sensory  infmmation,  make  cqitimal  decisions  based  on  the 
senamy  inputs  and  well-defined  objectives,  and  translate  those  decisions  into  actions,  is  a  guiding  and 
unifying  theme  for  basic  research  in  all  three  ditectirms:  control,  peicqrtual  systems,  and  communications 
and  computation.  The  technological  challenge,  for  example,  of  implementing  a  control  and  communication 
system  for  a  remote  tele-operated  vehicle  ova  a  tow  bandwidth  conununication  charmel  is  not  simply 
addressed  with  methods  already  available.  The  effective  design  of  such  a  system  stimulates  and,  in  oim, 
benefits  from  new  mathematical  formulations  and  analysis  of  problems  in  communication  theory  computa 
vision,  and  control. 

Research  on  intelligent  control  at  the  CICS  has  been  led  by  Professors  Brockett,  Ho,  Kushna  and  Mitta. 
Details  of  their  work  may  be  found  in  their  individual  repots  in  Section  92. 

Research  on  Learning 

Learning  research  at  CICS  in  the  last  few  years  has  been  primarily  theoretical  ratha  than  empirical  in 
characta.  Our  aim  was  to  develop  the  fundamental  princiiHes  and  techniques  for  learning  by  machines. 
Artificial  intelligence  (AI)  research  in  learning  has  been  typically  characterized  as  empirical  in  itature,  with 
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little  attention  paid  to  the  generality  and  scalability  of  the  techniques.  In  contrast,  our  thecvetical  research  is 
very  much  concerned  with  the  efficient  use  of  resources  (both  computational  time  and  data  are  the  resources 
considered),  and  with  the  breadth  of  applicabili^  (trying  to  learn  as  generally  as  possible). 

A  very  impotant  part  of  our  work  is  the  devetopment  and  mcpkxation  of  various  formal  models  for 
learning.  The  "distribution-fiee"  or  TAG"  model  (miginally  ixroposedby  Valiant)  has  been  one  of  our 
most  successful  models,  and  has  inspired  much  research.  In  this  model  of  concept  learning,  it  is  assumed 
that  there  is  an  unknown  probability  distribution  governing  the  generation  of  both  training  and  test 
examples;  this  rqjresents  the  "natural"  distribution  the  learner  faces.  The  learner  must  come  up  with  a 
hypothesis  that  correctly  classifies  an  example  drawn  randomly  from  this  distribution  with  high  probability, 
using  a  reasonable  (polynomial)  number  of  examples  and  a  reasonable  (polynomial)  computation  time. 

Another  direction  of  research  that  we  have  pursued  is  the  understanding  by  machines  of  simple 
environments.  We  may  wish  to  build  a  robot  that,  whoi  placed  in  such  an  environment,  can  build  up  a 
model  of  its  environment  by  experimentation  and  expknation.  Gt  may  have  a  variety  of  primitive 
operations  available  that  characterize  its  interface  to  its  environment,  such  as  walking,  manipulating,  etc.) 
We  have  investigated  what  can  be  done  with  deterministic,  finite-state  environments,  and  have  developed 
powerful  new  techniques  for  learning  such  environments. 

AiKXher  question  we  have  pursued  concerns  the  fundamental  iimitntif>n<  of  tnaming  _  what  can  we  say 
tdXMit  what  can  not  be  learned  efficiently?  Such  research  helped  to  clarify  the  boundary  between  what  is 
efficiently  kamable  and  what  is  not  efficiently  leamable  in  a  given  model;  giving  guidance  as  to  when  the 
learning  algorithms  may  need  additional  assumptions  or  tools.  For  example,  the  ability  to  actively 
experiment  with  an  environment  gives  the  learning  algorithm  greater  power  than  merely  being  able  to 
passively  observe.  Similarly,  certain  kinds  of  concqtts  may  be  efficiently  leamable  if  the  underlying 
probability  distribution  is  known  to  be  of  a  certain  type,  but  not  if  the  distributmn  is  arbitrary. 

We  have  also  investigated  the  effect  of  noisy  data  on  leamability.  In  many  cases,  it  can  be  shown  that  if  a 
certain  class  of  concepts  is  leamable  in  the  noise-fiee  case,  then  a  ^vj***"  amount  of  noise  in  the  data  can  be 
tolerated  as  well,  without  interfering  with  leamability.  Quantifying  the  amount  of  noise  that  can  be 
tolerated  is  a  significant  theme  in  our  research.  There  are  other  themes  to  our  research  that  reflect  concerns 
common  to  AI  as  well,  such  as  the  effect  of  the  knowledge  representation  techniques  on  leamability.  Here, 
however,  our  concerns  typically  reflect  the  impact  of  representation  on  computational  efficiency,  whereas  a 
typical  AI  igtptoach  would  be  concerned  with  representational  expressive  power  alone. 

Other  conemns  that  we  address  are  more  statistical  in  nature,  reflecting  the  fact  that  learning  is  a  gradual 
process  where  knowledge  is  accumulated  over  many  examines,  and  that  the  amount  of  data  required  to  learn 
dqtends  both  on  the  quality  of  the  data  and  on  the  prior  knowledge  available  about  the  unknown  coneqK  or 
phenomenotL  A  related  statistical  issue  we  have  investigated  relates  the  quality  of  "fit"  of  a  hypothesis  to 
the  data  versus  the  "complexity"  of  the  hypothesis;  this  is  a  fundamental  trade-off  in  many  real-life 
triplications. 

The  theoretical  work  in  machine  teaming  has  already  had  an  impact  on  practice,  in  that  empirical  researchers 
wiU  now  try  to  assess  the  adequacy  of  their  data  bases  for  the  kinds  of  coiKlusions  they  are  drawing,  by 
comparing  the  theoretical  bounds  on  the  amount  of  required  with  the  amount  available. 

Our  work  on  exploration  of  environments  is  fundamental  research  directed  towards  iqiidications  such  as 
auumomous  robots  expiring  unknown  environments.  While  our  research  makes  assumptions  (such  as  the 
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finite-state  assumption)  that  may  be  unrealistic  in  practice,  we  are  looking  to  relax  as  many  of  these 
assumptions  as  we  can,  to  match  theoretical  work  with  real  applications. 

We  expect  that  a  number  of  the  learning  algorithms  that  we  have  developed  for  theoretical  purposes  are 
likely  to  be  of  some  practical  utility.  For  example,  one  of  our  learning  algorithms  based  on  the  use  of 
Fourier  transforms  in  boolean  domains  has  inspired  some  researchers  to  apply  these  techniques  in  practice, 
with  good  results. 

Our  studies  of  the  effect  of  noise  on  leamability  may  have  utility  in  applications,  since  real-life  data  is 
rarely  "clean." 

Research  on  learning  at  the  CICS  has  been  led  by  Professors  Mitter,  Rivest,  and  Valiant  Details  of  their 
w(rt  may  be  found  in  their  individual  reports  in  Section  9.2. 

Control  of  Linear  and  Nonlinear  Systems 

A  primary  objective  of  the  research  in  the  Center  is  to  caimire  in  a  quantitative  and  precise  way  the 
fundamental  limitations  of  controller  design  in  the  presence  of  uncertainty  and  to  provide  a  consistent 
methodology  for  design.  The  uncertain  environment,  as  well  as  the  design  specifications,  should  reflect 
realistic  and  practical  siuiations.  In  that  regard,  we  have  developed  a  new  methodology,  now  known  as  the 
/}  design  methodology,  for  designing  robust  controllers  in  the  presence  of  bounded,  persistent,  but 
unknown  disturbances.  This  theory  has  received  considerable  attention  by  the  contrtd  community,  in  both 
academia  and  industry,  for  its  practical  significance  and  computational  advantages.  The  /i  thetay  parallels 
in  ^irit  the  Hoo  theory.  However,  it  differs  in  the  sense  that  it  is  the  only  theory  that  allows  time  domain 
specifications  in  the  presence  of  unknown,  bounded  disturbances. 

The  computation  of  /i  optimal  solutions  involves  solutions  of  semi-infinite  linear  programs.  In  this 
reflect,  we  have  developed  methods  for  approximating  these  problems  by  utilizing  the  specific  structure  of 
these  LPs.  The  approximation  is  in  the  form  of  lower  and  upper  qrproximations  of  the  problem  that 
converge  to  the  exact  value  in  the  limit 

A  crucial  development  of  the  /i  theory  was  in  the  area  of  robust  stability.  There,  it  was  shown  that 
conditions  on  the  li  norm  of  the  systems  transfer  function  {Hovide  tests  for  stability  robustness  in  the 
presence  d  plant  perturbations.  These  conditions  were  shown  to  be  non-conservative,  and  can  be  optimized 
using  the /i  theo^  to  maximize  the  stability  margin. 

The  robust  performance  problem  in  the  presence  of  structured  perturbations  can  be  solved  by  satisfying 
conditions  in  terms  of  a  scaled  /i  norm.  We  have  shown  that  these  ctmditions  are  related  to  the  qwctial 
radius  d  certain  matrices  that  capture  the  structure  information.  Using  these  ideas,  we  hope  to  develop 
software  by  which  one  can  synthesize  controllers  that  deliver  robust  performance.  Also,  we  are  developing 
software  utilizing  Matlab,  that  will  allow  us  to  test  the  methodology  on  practical  problems.  Preliminary 
work  has  been  done  tq)plying  these  ideas  to  the  X-29  aircraft. 

Another  area  of  research  concerns  infinite-dimensional  systems.  Most  systems  are,  in  reality,  infinite- 
dimensional,  i.e.  governed  by  partial  differential  equations.  Examples  of  such  systems  are  delay  systems. 
This  motivated  us  to  devek^  a  theory  that  allows  designing  controllers  for  infinite-dintensional  systems 
using  finite  dimensional  techniques.  This  was  drnie  by  q>praximating  the  plant  in  qtecific  frequency  ranges 
(usually  compact  sets)  by  a  finite-dimensional  plant  A  robust  controller  is  then  designed  based  on  this 
finite  dimensional  plant  This  design  has  to  be  done  in  a  very  qtecific  way,  since  otherwise  the  resulting 
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controller  may  not  even  be  a  stabilizing  controller.  We  have  shown  that  we  can  design  Hoo  controllers  that 
deliver  a  performance  within  a  small  deviation  from  the  best  achievable  poformance.  This  jxovided  a  readily 
computable  method  for  obtaining  //oo  sub-optimal  controllers  for  infinite-dimensional  plants.  In  addition, 
exact  solutions  for  the  /i  and  //«>  problems  in  the  presence  of  delays  (commensurate)  have  been  fiunished. 

Motivated  by  the  many  applications  of  multirate  controllers,  we  have  studied  the  problem  of  multirate 
systems  in  the  context  of  robust  control.  In  the  case  of  synchronous  sampling,  we  have  provided 
computable  solutions  for  the  /i  robust  performance  problem.  We  have  shown  that  the  solutions  for  this 
I»Dblem  are  no  harder  to  obtain  than  are  those  of  the  standard  /i  problem.  The  linear  programs  are  in  general 
of  higher  dimension.  In  the  Hoo  context,  we  have  shown  that  the  solution  of  this  problem  can  be  obtained 
by  a  convex,  finite  dimensional  programming  problem  which  is  readily  computable. 

In  the  case  of  asynchronous  sampling,  we  have  done  preliminary  studies  showing  that  such  systems  can  be 
approximated  by  a  sequence  of  slowly  varying  synchronous  systems.  This  problem  is  still  under 
considoation. 

We  have  developed  a  methodology  based  on  linear  theory  for  designing  robust  controllers  for  slowly-time- 
varying  systems,  in  which  the  dimension  of  the  system  is  not  known  to  be  Hxed.  The  development  is 
input/output  and  has  immediate  applications  to  adaptive  control.  The  reason  is  that  in  adaptive  control,  the 
estimator  provides  estimates  that  form  a  slowly-time  varying  system,  to  which  a  controller  is  designed.  We 
have  shown  that  the  closed  loop  performance  can  be  direedy  related  to  the  frozen-time  performance  of  the 
system  and  hence,  providing  consmictive  methods  for  designing  ad^tive  controllers  that  achieve  a  certain 
degree  of  stability  robustness. 

A  basic  problem  in  robust  control  theory  is  recognizing  whether  one  needs  adaptive  time  varying  controllers 
in  a  specific  design  inoblcm.  For  this  purpose,  we  have  studied  the  advantages  gained  in  using  time- varying 
controllers  for  the  disturbance  rejection  and  robust  stabilization  problems  in  the  presence  of  bounded  inputs. 
We  have  shown  that  time-varying  controllers  offer  no  advantages  in  these  two  problems  in  the  case  where 
the  plant  is  linear  time-invariant.  Similarly  for  nonlinear  smooth  controllers.  The  problem  is  still  open 
whoi  the  uncertainty  is  structured  in  nature. 

Research  on  the  control  of  linear  and  nonlinear  systems  at  the  CICS  has  been  led  by  Professors  Brockett, 
Dahleh.  Fleming,  Kushner,  Mitter,  and  Verghese.  Details  of  their  work  may  be  found  in  their  individual 
reports  in  Section  9.2. 

Computation 

It  is  often  said  that  parallel  computation  offers  the  best  hope  for  solving  very  large  problems.  However, 
there  is  increasing  awareness  that  making  this  hope  a  reality  is  very  challenging,  even  for  problems  with 
naturally  parallclizable  structure.  Specialized  processor  arrays  can  be  very  effective  for  some  highly  regular 
problems  but  often  caimot  cope  efficiendy  with  the  sparsity  structures  that  characterize  many  large 
numerical  problems.  In  shared  memory  machines  the  number  of  processors  is  limited  to  a  few  tens,  so  the 
potential  for  large  speedup  and  large  problem  solution  is  also  limited.  In  massively  parallel  message¬ 
passing  machines  (e.g.  hypeicube-like  architectures  with  thousands  of  processors),  a  principal  bottleneck  is 
the  communication  cost  In  particular,  solving  large  numerical  problems  in  reasonable  time  requires  fine 
grain  parallelization  and  an  accordingly  large  number  of  message  exchanges  between  processors. 
Furthermore,  when  using  iterative  methods,  there  is  a  substantial  penalty  for  synchronizing  a  large  number 
of  processus  at  each  iteration. 
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To  alleviate  the  communication  bottleneck  in  message-passing  systems,  it  is  important  to  develop  better 
communication  algorithms,  and  to  understand  the  nature  and  impact  of  communication  and  synchronization 
delays. 

Working  along  these  lines,  we  have  been  conducting  research  in  two  broad  areas: 

(i)  development  of  optimal  routing  algorithms  for  generic  conununication  problems  in  hypeicube  and 
other  architectures; 

(ii)  quantifying  the  advantages  and  limitations  of  asynchronous  algorithmic  implementations  tlmt  may 
avoid  the  synchronization  penalty  inherent  in  synchronous  iterative  methods. 

There  are  a  number  of  generic  communication  problems  that  arise  fluently  when  using  iterative  and  other 
algorithms  in  message-passing  machines.  For  example,  we  may  want  to  send  a  message  from  each 
processor  to  every  other  processor.  This  problem,  called  multinode  broadcast,  arises  in  linear  iterations  of 
the  form  x:^Ax  +  b.  If  we  assume  that  there  is  a  different  processor  updating  each  coordituue  of  x,  and 
that  the  matrix  A  is  dense,  then  at  the  end  of  an  iteration,  there  is  a  need  for  every  processor  to  send  the 
updated  value  of  its  coordinate  to  every  other  processor. 

Algorithms  for  solving  the  multinotte  broadcast  and  other  related  communication  problems  have  recently 
been  developed  for  several  popular  architectures.  These  algorithms  execute  in  time  that  grows  at  the 
optimal  rate  with  the  number  of  processors,  and  are  often  optimal  in  that  they  terminate  in  the  minimum 
possible  number  of  time  steps.  The  underlying  assumption  here  is  that  each  message  requires  unit  time  for 
transmission  on  any  link  of  the  processor  network,  and  that  each  processor  can  transmit  and  receive  a 
message  simultaneously  on  all  of  its  incident  links. 

Current  research  is  directed  towards  relaxing  these  assumptions  and  towards  developing  optimal  algorithms 
for  a  variety  of  additional  communication  problems.  This  worit  also  tries  to  evaluate  the  impact  of 
communication  on  the  solution  of  large  problems.  Fm*  example,  it  is  simple  to  show  that  by  using  a 
hypercube  networit  of  processors  and  optimal  communication  algorithms,  most  of  the  basic  operations  of 
numerical  linear  algebra  -  i.e.  inner  product,  matrix-vector  multiplication,  matrix-matrix  multiplication, 
power  of  a  matrix,  etc.  --  can  be  executed  in  parallel  in  the  same  order  of  time  as  when  communication  is 
instantaneous.  In  some  cases  this  is  also  possible  when  the  processes  are  connected  with  a  less  powerful 
networit  such  as  a  square  mesh. 

In  asynchronous  implementation  of  parallel  iterative  algorithms,  processors  are  not  required  to  wait  to 
receive  all  messages  generated  during  the  previous  iteratioi.  Rather,  each  processor  is  allowed  to  keep 
iterating  on  its  own  component  at  its  own  pace.  If  the  current  value  of  the  component  updated  by  some 
other  processor  is  not  available,  then  some  outdated  value  received  at  some  past  time  is  used  instead. 
Furthermore,  processors  are  not  required  to  communicate  their  results  after  each  iteration  but  only  once  in  a 
while.  Thus,  some  processors  may  compute  faster  and  execute  more  iterations  than  others,  some  processors 
may  communicate  more  frequently  than  others,  and  the  communication  delays  may  be  substantial  and 
unpredictable. 

The  potential  advantages  frnm  asynchronous  execution  are  implementation  flexibility,  and  convergence 
acceleration  due  to  a  reduction  of  the  synchronization  penalty,  frideed,  recent  experimental  wok  as  well  as 
theoretical  analysis  have  shown  that  for  important  classes  of  problems,  asynchronous  implementations  lead 
to  substantial  gains  in  speed  of  convergence  over  their  synchronous  counterparts.  A  drawback  of 
asynchronous  iterations  is  that  they  may  diverge  for  some  problems  where  their  synchronous  counterparts 
are  guaranteed  to  converge.  Nevertheless,  a  large  number  of  interesting  and  mathematically  challenging 
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results  have  been  developed  showing  that  certain  classes  of  important  algorithms  retain  their  desirable 
convergence  properties  in  the  face  of  asynchronism. 

In  the  area  of  simulated  annealing,  we  have  made  several  major  contributions.  The  paper  by  Tsitsiklis  may 
well  be  the  definitive  paper  on  simulated  annealing  for  finite  state  spaces.  The  paper  by  Gelfand  and  Mitter 
is  the  first  analysis  of  simulated  annealing  in  unbounded  domains  and  continuous  state  space.  The  papers 
by  Holley-Stroock  and  Diaconis-Stroock  give  interesting  connections  between  the  spectral  gap  and  optimal 
cooling  rates  in  simulated  annealing. 

Another  area  of  research  concerns  continuous  computation.  Although  many  of  the  classical  algorithms 
involving  descent  can  be  interpreted  as  using  the  integral  curves  of  differential  equations  to  guide  the  flow 
of  the  successive  approximations,  the  digital  implementation  of  such  algorithms  usually  seeks  to  replace 
the  continuous  descent  algorithms  with  one  which  involves  large  (discrete)  step  sizes.  This  usually  leads 
to  faster  convergence.  On  the  other  hand,  it  has  been  reported  in  the  literature  that  waen  using  descent 
methods  for  function  minimization  it  is  better  to  use  continuous  descent  if  the  function  is  so  complicated 
that  it  can  not  be  well  approximated  by  a  quadratic.  Meveover,  as  the  technology  associated  with  analog 
VLSI  grows  more  mature,  direct  analog  simulation  is  becoming  feasible  in  more  cases. 

Several  researchers  in  the  Center  have  pursued  ideas  along  these  lines.  On  one  hand  we  now  know  that 
arbitrary  automata  can  be  realized  by  assigning  the  states  of  the  automata  to  equilibrium  points  of  a  forced 
differential  equation  x'  =  f{xM)  and  that  we  can  always  take /  to  be  quadratic  in  x.  We  have  also  shown  that 
a  variety  of  techniques  from  linear  algebra  can  be  generated  by  flows  on  the  space  of  matrices. 

Eigenvalues,  singular  values,  eigenspaces,  etc.  can  be  generated  in  this  way.  In  his  thesis,  Faybusovich 
has  undertaken  a  systematic  study  of  linear  programming  from  this  point  of  view.  This  work  also  makes 
contact  with  the  literature  on  training  neural  networks  and  the  problem  of  learning  principal  components. 

Research  on  computation  at  the  CICS  has  been  led  by  Professors  Bertsekas,  Qeighton,  Goldwasser,  Micali, 
Strang,  and  Tsitsiklis.  Details  of  their  work  may  be  found  in  their  individual  reports  in  Section  9.2. 

Research  on  Communication  and  Distributed  Computation 

It  is  widely  recognized  that  it  is  technologically  difficult  and  c^jerationally  ill-advised  to  build  complex 
systems  that  cover  large  geographical  areas,  and  are  controUed  in  a  centralized  way.  Because  of  this, 
distributed  control  systems  have  become  increasingly  iraptntant,  and  the  corresponding  distributed 
algorithms  have  been  the  subject  of  intensive  study.  At  the  same  time  there  has  been  continuing  research 
on  communication  networks  that  use  distributed  algorithms  and  also  form  an  integral  part  of  distributed 
intelligent  systems. 

Research  on  communication  networks  is  relevant  to  intelligent  systems  in  three  important  ways.  First, 
distributed  intelligent  systems  contain  networked  communication  as  an  essential  ingredient,  and  the 
performance  of  the  system  depends  critically  on  the  poformance  of  the  network.  Second,  and  more 
important,  the  control  of  a  communicadon  network  is  a  problem  in  which  classical  control  is  inadequate 
and  an  approach  more  like  that  being  developed  for  intelligent  control  is  required.  Third,  and  perhaps  most 
important,  understanding  the  problems  of  (hstributed  estimation,  and  distributed  recovery  from  error  events 
is  cridcal  to  establishing  conceptual  foundadons  for  the  intelligent  control  of  large  distributed  systems. 
These  problems  appear  in  a  pardcularly  pure  and  unencumbered  form  in  the  study  of  networks.  A  variety  of 
distributed  communicadon  issues  are  now  being  invesdgated  such  as  failure  recovery,  distributed  roudng. 
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flow  control,  aiui  broadband  multiaccess  communication.  With  communication  speeds  increasing  by  orders 
of  magnitude  in  the  near  future,  the  conceptual  framework  for  addressing  these  issues  is  likely  to  evolve 
considerably. 

Research  on  communication  and  distributed  computation  at  the  CICS  has  been  led  by  Professors  Bertsekas, 
Gallager,  Humblet,  and  Tsitsiklis.  Details  of  their  woik  may  be  found  in  their  individual  reports  in  Section 
9.2. 


9.2  Researchers'  Reports 
Dimitri  Bertsekas 

The  work  led  by  Professor  Bertsekas  has  focussed  on  two  areas:  computational  methods  for  optimization 
problems,  and  parallel  and  distributed  algorithms. 

In  the  area  of  computational  methods  for  optimization  problems.  Professor  Bertsekas  and  his  co-workers  have 
investigated  projection  methods  for  constrained  optimization,  coordinate  ascent  and  decomposition  algorithms 
for  large-scale  linear  and  nonlinear  programming,  interior  point  methods  for  linear  programming,  and  linear 
and  convex  network  optimization  methods.  This  woric  has  aimed  primarily  at  the  development  of 
methodology  for  solving  large-scale  problems.  Many  of  these  problems  have  structures  that  are  amenable  to 
decomposition  and  parallelization,  and  they  often  involve  a  graph  structure  and/or  a  dynamic  system  structure. 
In  the  former  case,  the  number  of  nodes  and  arcs  is  often  very  large,  while  in  the  latter  case,  problems  of  large 
dimension  often  arise  because  of  the  underlying  time  and/m*  space  disoetizaiion  of  a  differential  equation.  Both 
graph  structures  and  dynamic  system  structures  have  been  extensively  considered  in  this  work. 

In  the  area  of  parallel  and  distributed  algorithms.  Professor  Bertsekas  and  his  co-workers  have  investigated 
communication  issues  in  interconnection  networks  of  parallel  processors.  In  particular,  they  have 
investigated  some  basic  communication  problems  that  arise  in  numy  algorithmic  contexts,  auch  as 
muliitKXie  broadcast  and  total  exchange.  The  effect  of  randomness  of  packet  length  arxl  arrival  time  of 
messages  was  quantified  in  the  context  of  some  common  architectures  such  as  hypercube  and  mesh.  They 
have  also  established  general  frameworks  for  convergence  analysis  of  distributed  asynchronous  iterative 
processes,  and  they  analyzed  several  specific  algorithms  relating  to  shortest  paths,  dynamic  programming, 
gradient  methods,  and  netwodt  optimization. 

Roger  Brockett 

Doting  this  period,  ProfesstX'  Brockett  and  his  students  perftumed  experimental  and  theoretical  work  in  the 
areas  oi  robotics,  computer  vision  and  analog  computation.  Highlights  of  the  work  on  robotics  include  the 
use  of  new  geometrical  methods  (harmonic  mappings)  for  the  design  of  kinanaiics  (Frank  Paric),  and 
Bayesian  inversion  algorithms  for  the  interpretation  of  tactile  sensing  data  (William  Nowlin).  In  the  area  of 
computer  vision  the  work  included  the  development  of  real-time  algorithms  for  the  motion  from  mmnents 
I»dblem  (Morris  Lee),  and  the  development  of  robust  matching  algorithms  based  on  minimal  stress 
interpolation  (Ed  Rak).  Developments  in  analog  computing  include  the  proof  that  a  variety  of 
combinatorial  problems  can  be  solved  by  a  relatively  simple  class  of  ordiruiry  differential  equations,  and  that 
it  is  possible  to  use  differential  equations  to  simulate  finite  automata  in  a  reasonably  straight  forward  way. 
The  wtnk  of  Faybusovich  clarified  the  role  of  interior  trajectory  methods  for  linear  programming  and 
provided  new  algorithms.  New  work  on  the  design  of  neural  networks  based  on  the  propagation  of  the 
conditional  density  shows  that  the  double  bracket  equation  can  be  thought  of  as  providing  the  optimal  data 
for  solving  certain  types  of  sorting  problems  involving  noisy  data.  Finally,  the  properties  of  a  new  class  of 
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filters  that  discriminate  on  the  basis  of  the  way  the  energy  in  a  signal  is  distributed  among  various 
subspaces  were  defined  and  studied.  It  was  shown  that  these  adaptive  subspace  filters  can  be  used  to  solve 
problems  in  system  identification  and  signal  detection, 

Munther  Dahleh 

During  this  period.  Professor  Dahleh  was  involved  in  two  major  problems.  The  first  was  the  development 
of  a  robust  control  methodology  in  which  realistic  time-domain  q)eciA:ations  can  be  incorporated 
systematically  in  the  presence  of  plant  and  input  uncertainty.  Algorithms  based  on  linear  programming 
computations  were  developed.  This  methodology  was  extended  to  incorporate  Hybrid  systems,  i.e. 
continuous-time  systems  with  digital  controllers,  otherwise  known  as  sampled-data  systems.  The  second 
problem  was  the  study  of  system  identification  in  the  presence  of  bounded  but  unknown  noise.  In 
particular,  the  objective  was  to  study  the  problem  of  worst-case  idoitification.  The  motivation  behind  this 
problem  is  the  desire  to  use  a  given  model  in  robust  control  applications.  In  this  case,  the  model  should 
approximate  the  actual  process  in  a  strong  sense,  which  is  not  generally  possible  with  the  traditional 
identification  schemes.  Results  in  this  direction  showed  that  one  can  devise  fairly  simple  algorithms  that 
guarantee  a  small  error  between  the  estimate  and  the  process  for  a  fairly  general  class  of  model  sets,  {xovided 
that  the  inputs  are  chosen  appropriately.  A  study  of  the  complexity  of  such  algorithms  was  conducted. 

Professor  Dahleh  had  extensive  interactions  with  Dr.  Normal  Coleman  and  his  group  at  ARDEC,  through 
various  tutorials  and  workshops.  This  interaction  between  LIDS  and  ARDEC  took  a  big  step  ahead  when 
the  center  hired  Dr.  Dragan  Obradovich  as  a  postdoctoral  associate,  supervised  by  Prof.  Dahteh.  Since  then, 
both  have  been  heavily  involved  in  a  controller  design  for  a  gun  that  will  be  mounted  on  top  of  an  aircrafL 
This  control  problem  highlighted  new  and  interesting  directions  for  research,  particularly  in  the  problem  of 
identification  within  uncertainty. 

Wendell  H.  Fleming 

During  this  period.  Professor  Fleming  and  his  students  and  postdoctoral  fellows  conducted  analytical  and 
numerical  studies  of  topics  in  applied  probability,  stochastic  control  and  nonlinear  estimation.  In  tqpplied 
probability,  sharper  versions  of  Freidlin-Wentzell  large  deviations  formulas  in  the  ftxm  of  WKB-type 
asymptotic  scries  were  obtained.  A  combination  of  probabilistic  and  partial  differential  equations  (PDE) 
methods  were  used.  In  stochastic  control,  fuiite  difference  approximations  to  dynamic  progranuning  PDEs 
were  studied,  for  models  with  a  special  strucbue  which  facilitates  proofs  of  convergence  of  optimal  feedback 
control  policies.  Risk-sensitive  stochastic  control  problems  and  their  connectimi  with  robust  nonlinear 
control  were  studied,  using  viscosity  solution  methods  for  nonlinear  PDEs.  In  addition,  stochastic  control 
problems  were  considered  in  a  convex  duality  framework. 

Nonlinear  filtering  problems  were  studied  in  cases  where  a  piecewise  one-to-<me  function  of  a  system  state 
plus  low  intensity  white  noise  is  observed.  Sequentitd  hypothesis  tests  to  discriminate  among  regions  in 
which  the  observation  function  is  one-to  one  were  developed.  The  small  observation  ndse  analyses  {oovide 
justification  for  ad  hoc  approximate  filtering  techniques  such  as  extended  Kalman  Filters  and  their 
extensions. 

Robert  Gallager 

Professor  Robert  Gallager  worked  on  an  information  theoretic  investigation  of  broad  band  communication 
channels  (both  point-to-point  and  multiaccess).  Given  an  energy  constraint,  the  reliability  function  of  such 
point-to-point  channels  has  been  found  exactly  in  the  limit  of  an  unbounded  number  of  degrees  of  freedom. 
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These  results  have  been  extended  to  multiaccess  channels,  where  results  shed  light  both  on  the  issue  of 
multiaccess  decoding  and  on  the  relationship  between  spread  spectrum  and  random  access  communication. 
Professor  Gallager  also  wwked  on  performance  guarantees  and  congestion  control  in  high  speed  networks, 
and  (with  John  Spinelli)  on  failure  recovery  protocols  in  networks,  investigating  methods  for  a  computer 
network  to  provide  reliable  communication  when  some  of  its  components  may  fail. 

Stuart  Geman,  Basilis  Gidas,  Ulf  Grenander,  and  Donald  McClure 

The  Brown  University  Pattern  Theory  Group's  wok  has  concentrated  on  the  development  of  rigorous 
mathematical  foundations  for  problems  in  computer  vision  and  image  analysis.  Principals  of  the  group 
include  S.  Geman,  B.  Gidas,  U.  Grenander  and  D.  McClure.  A  general  theme  of  the  research  is  to  guide  the 
development  of  algorithms  and  the  solution  of  computational  problems  by  carefully  articulated 
mathematical  models. 

Probabilistic  models  and  i»inciples  of  Bayesian  infoence  are  cornerstones  of  the  group's  approach  to 
problems  in  image  analysis.  Predominaru  among  the  probabilistic  models  used  are  Markov  Random  Fidds 
(MRFs).  There  are  several  important  instances  of  the  use  of  Markov  processes  in  computer  vision  and 
image  processing  by  others  in  the  early  1980s  and  earlier.  This  group  played  an  important  role  in 
fashioning  the  rigorous  mathematical  framework  on  which  the  MRF/Bayesian  approach  rests,  starting 
around  1983.  Early  contributions  included: 

(0  the  crystalization  of  the  Markovian  framework, 

(ii)  the  recognition  of  the  Markov-Gibbs  correspondence  and  its  importance  for  modeling  and  for 
designing  and  understanding  computational  aiqnoaches, 

(iii)  the  definition  of  Monte>Carlo  methods  for  sampling  from  Gibbs  distributions,  and 

Qv)  the  first  proofs  of  convergence  and  diver^ce  of  simulated  annealing  as  a  method  of  global 

qitimization. 

These  advances  encouraged  the  use  of  well-founded  probabilistic  and  statistical  methods  in  image  analysis. 

The  focus  of  the  group's  research  has  evolved  from  the  early  phase  that  emphasized  low-level  image 
processing  and  simple  lattice-based  MRFs  to  a  current  focus  on  the  development  of  richer  mathematical 
models  and  high-level  computer  vision  problems.  The  completed  research  has  included  basic  foundational 
issues  for  Markov  processes,  important  frx'  understanding  how  broadly  the  use  of  Markov  models  can  be 
justified,  as  well  as  preliminary  work  on  probabilistic  models  for  shapes  which  exhibit  global 
structureAegularity  and,  at  the  same  time,  high  degrees  of  variability  from  instance  to  instance  -  e.g.  a 
human  hand  and  other  biological  forms. 

The  groups  research  has  emphasized  the  development  of  mathematical  tools,  smne  motivated  by  intrinsic 
mathematical  interest  and  some  by  applications.  The  modeling  qrptoaches  have  been  strongly  influenced 
by  qq>lications.  In  completed  research,  the  utility  of  MRFs  for  modeling  low-level  regularities  in  images 
was  driven  by  triplications  to  automated  industrial  inq;)ection  and  emission  emnputed  tomography.  The 
triplications  stimulate  the  mathematical  research  and,  in  turn,  the  algorithms  derived  from  the  mathematical 
analysis  provide  new  methodological  approaches  for  the  triidkations. 

In  the  area  of  foundations,  a  comprehensive  presentation  of  general  pattern  theory  (by  Grenander  and  Miller) 
has  recently  been  completed  and  is  in  the  process  of  final  editing  for  publication.  The  volume  describes  the 
unified  theory  for  regular  structures  developed  by  the  author  over  the  past  twenty-five  years.  In  very  recent 
work  (Gidas  and  Hudson;  Raphael),  the  issue  of  how  generic  so-calted  hidden  Maikov  models  (HMMs)  are 
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has  been  addressed.  This  woik  has  made  extensive  use  of  Maikov  {xocesses  as  models  for  grey-ievel 
distributions,  boundaries,  textures,  isotope  concentration  mt^,  and,  more  recently,  for  sh^tes  in 
reconstruction  and  recognition  experiments.  In  these  settings,  as  well  as  in  Markov-based  speech 
recognition,  the  actual  observations  are  functions  of  a  Markov  process,  thereby  "hidden  Markov,"  with  a 
rich  and  complex  dependency  structure.  At  the  same  time,  the  basic  building  blocks  for  the  process  exhibit 
simple,  local  Maricovian  dependence.  Therein  lies  the  utility  of  HMMs.  Key  results  cover 

(i)  the  approximation  of  an  arbitrary  stationary  process  with  finite  state  space  and  indexed  on  a 
d-dimensional  lattice  by  a  nearest-neighbor  HMM,  in  the  topology  of  weak  convergence,  and 

(ii)  consistency  results  for  nonparametric  estimation  algorithms  to  fit  HMMs  to  observations  of  an 
arbitrary  stationary  process. 

In  the  area  of  learning  and  parameter  estimation,  an  important  step  in  applications  of  the  group's  fiamework 
is  the  estimation  or  "learning"  of  Maritov  random  fields  from  (possibly  degraded)  data.  The  estimation 
problem  for  Maikov  random  fields  generalizes  in  a  fundamental  way  that  of  time  series  aixl  has  motivated 
the  design  of  computationally  efficient  algorithms  (by  Almeida  and  Gidas).  It  has  also  led  to  interesting 
mathematical  problems  for  consistency  and  asymptotic  normality,  whose  partial  resolution  (by  Geman; 
Geman  and  Gidas;  and  Grcnandcr)  has  involved  large  deviation  techniques,  a  new  variational  principle,  and 
tools  from  statistical  mechanics.  The  nonparametric  learning  problem  for  large,  complex  systems  (e.g. 
neural  nets)  was  also  considered  (by  Comets  and  Gidas):  attempts  to  estimate  large  numbers  of  parameters 
in  neural  networks  are  seen  in  the  context  of  standard  nonparametric  estimation  and  regression,  thereby 
drawing  on  a  substantial  and  well-established  theoretical  base  and  exposing  some  fundamental  limitations 
implied  by  the  "bias-variance"  dilemma  and  other  tradeoffs. 

In  the  area  of  algorithm  design,  analysis  and  optimization,  the  bask  stochastk  relaxation  algorithms 
developed  in  the  group's  earlier  work  proved  to  be  broadly  applicable,  even  if  not  highly  efficient,  and 
stimulated  extensive  mathematical  research  on  convergence  and  conqwtationai  complexity.  In  recent 
research,  motivated  by  the  need  for  algorithms  well-suited  to  more  highly  structured  models,  group 
members  have  developed: 

(i)  a  multistage/mutircsolution  algorithm  based  on  the  Langevin  equation  (by  Amit,  Grenandcr,  and 
Piccioni:  Amit  and  Piccioni):  and 

(u)  a  jump-diffusion  algorithm  for  recognizing  an  a  priori  unknown  number  of  deformable  objects 

(e.g.  mitochondria)  in  a  scene  (e.g.  an  election  microgTsph)  (by  Manbeck). 

The  latter  algorithm  has  been  implemented  on  a  NEWS  architecture  SIMD  parallel  system,  enabling  real¬ 
time  video  monitoring  of  its  behavior.  A  fundamental  theoretical  issue  addressed  by  Manbeck,  and  a 
recurring  analytical  question,  is  to  prove  the  eigodidty  of  the  associated  Maikov  process.  Virtually  all  of 
the  computational  worit  by  our  group  has  dealt  with  questions  of  convergence  and  efficiency  of  simulation 
and  optimization  algmithms.  In  addition,  leguhu’  interactions  with  visiting  scientists  have  stimulated 
important  contrUnitions  by  groups  in  Rome,  Paris  and  Taipei  on  convergence  properties  of  Monte-Carlo 
procedures  for  stunpling  from  Gibbs  distributions. 

In  the  area  of  deformable  templates  and  object  recognition,  the  group  has  explored  a  model-based  iqqxoach 
to  nonrigid  object  recognition.  The  required  models  must  address  both  the  global  structural  characterization 
of  an  object  and  the  class  of  transformations  that  define  the  possiUe  presentations  of  the  object  It  has 
proposed  a  class  of  such  models  (Comets  and  Gidas;  Amit  Grenander,  and  Pkeioni),  whkh  it  calls 
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defcHinable  templates,  and  has  perfonned  preliminary  experiments  to  test  their  utility  in  a  variety  of 
tq)plications  (Comets  and  Gidas;  Amit.  Grenander,  and  Piccioni;  Manbeck).  The  study  of  deformable 
templates,  and  associated  algorithms  for  template  matching  in  objea  recognition  tasks,  has  stimulated  a 
number  of  theoretical  investigadons  (Amit;  Gidas  and  Hudson;  Amit  and  Piccioni),  which  have  yielded 
results  on  diverse  topics.  These  include  some  new  coarse-to-flne  type  methods  for  Monte  Carlo 
simuladons,  and  the  aforemrationed  result  establishing  the  density,  among  all  stationary  processes  in 
artutrary  dimensions,  of  hidden  Markov  models. 

In  the  area  of  ill-posed  inverse  problems,  the  groiq>  continued  to  study  ill-posed  inverse  problems  using 
stochastic  regularizadon  and  Bayesian  inference,  focusing  on  (plications  to  medicine  and  astronomy.  The 
work  by  Chow,  Gidas,  Grenander,  Hudson,  Keenan,  Raphael,  and  Toneao  involves  single  photon  emissicMi 
computed  tomography.  A  comprehensive  physical  model  was  developed,  which  includes  corrections  for 
attenuation  and  scattering;  all  experiments  involve  either  real  or  physical  phantom  data.  In  addition,  scaling 
properties  of  the  regularization  term  are  derived  by  computing  the  continuum  limit  of  the  associated  prior 
distribution.  In  contrast,  other  worir  addressed  a  "dual”  of  the  tomogrqthic  reconstruction  problem; 
determining  the  radiation  dosage  and  treatment  desi^  necessary  to  achieve  or  approximate  as  closely  as 
possible  a  specified  uptake  distribution. 

In  the  area  of  image  processing  and  recovery  of  low-level  structure,  the  group’s  previous  work  in  machine 
vision  has  emphasized  mathematical  methods  for  "low"  and  "medium"  level  image  processing  tasks,  such  as 
those  involving  the  ill-posed  inverse  problems  discussed  above.  Models  and  associated  computational 
algorithms  have  been  developed  for  a  number  of  potential  qjplications,  including  boundary  detection  and 
artifact  removal  for  film  restoration  (Geman,  Manbeck,  and  Mcaute),  shape  from  shading  (Grenander  and 
Miller),  and  iihage  representations  with  bivariate  splines  (Toneao).  Groiq>  members  explored  the  use  of 
qxuial-temporal  MRFs  for  the  problem  of  restoring  and  enhancing  video  sequences  d^raded  by  intra-fiame 
artifacts  (e.g.  dirt  on  the  original  film)  and  by  fiame-to-firame  artifacts  (e.g.  scnttches).  The  inclusion  of 
modon-compensated  inter-frame  bonds  in  the  underiying  MRF  graph  structure  enables  the  expected 
temporal  continuity  to  detea  and  remove  noise  and  arti&cts. 

Several  recent  books  and  review  articles  describe  the  qtatial  stadtics/Bayesian  firamework  for  image  analysis. 
The  two  volumes  by  Grenander  and  Miller,  and  Hwang,  Hwang-Ma,  and  Shea,  contain  a  systematic 
treatment  of  general  pattern  theory,  mathematical  Judies  of  simulation  and  optiinization  algorithms,  and  a 
host  of  qtplications,  while  the  paper  by  Geman,  Geman,  and  McClure  contains  a  detailed  exposition  of 
various  inverse  problems  in  image  analysis,  approaches  to  these  problems  within  the  groups  framework  and 
by  alternative  methods,  and  basic  properties  of  stochastic  relaxation  and  simulated  annealing.  Fiiudly,  the 
botdt  by  Grenander  is  a  comprehensive  exposition  d  mattematical  tools  and  properties  concerning 
Metropolis-type  Monte-Carlo  simulation  algorithms  and  simulated  aiuiealing  in  the  context  of  the  general 
theory  of  stochastic  processes  and  Markov  chains. 

Y.C.  Ho 

During  this  period,  the  support  of  the  center  for  intelligent  contrtd  helped  to  establish  the  subject  of 
Discrete  Event  Dynamic  Systems  (DEDS)  as  a  full-fledged  intellectual  discipline.  Professor  Y.  C.  Ho's 
group  at  Harvard,  chiefly  responsible  for  creating  the  subject,  accomplished  nine  PhD.  theses,  three 
published  bodes,  three  awards  for  best  paper  or  bode,  one  videotrpe  tutorial,  one  new  international  journal 
on  DEDS,  two  conferences  on  DEDS  sponsored  by  NSF,  63  invited  talks,  three  plenary  addresses  at  world 
congresses,  one  IEEE  Control  Engineering  and  Science  Field  Award,  and  one  membership  in  the  National 
Academy  of  Engineering.  There  were  three  references  to  DEDS  as  a  discipline  for  the  90s  in  white  psqpers  by 
SIAM.  ORSA  and  IEEE. 
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During  the  period  covered  by  this  report,  the  following  theses  were  written:  Equivalence  methods  in  the 
perturbation  analysis  of  queueing  networics  (Paul  Glasserman);  Three  topics  on  perturbetian  analysis 
(PiiQOz  Vakili);  Optimization  of  queueing  systems  using  perturbation  analysis  (Kfichael  C.  Fu);  Strong 
consistency  of  perturbation  analysis  estimates  (Jian-Qiang  Hu);  Comparison  methods  bx  scheduling  control 
of  multiclass  queues  (Gerald  E.  Feigin);  Extended  perturbation  analysis  of  discrete  event  dynamic  systems 
(Shu  Li);  Performance  gradient  estimation  for  very  large  Markov  Chains  (Bin  Zhang). 

Discrete  Event  Dynamic  Systems  (DEDS)  constitute  an  important  topic  of  control  system  study. 

Examples  of  DEDS  range  from  large  international  airports,  to  computer  communication  networks,  to 
manufacturing  plants,  logistics  and  service  systems,  and  C^I  systems.  The  design,  performance  evaluation, 
and  operations  management  of  such  man'tnade  systems  are  a  fertile  field  of  research.  The  Harvard  group, 
under  the  direction  of  Professor  Ho,  initiated  a  systematic  study  of  such  inoblems,  resulting  in  the  creation 
of  the  areas  of  perturbation  analysis,  parallel  simulation,  and  ordinal  optimization.  These  techniques 
promise  to  usher  in  a  new  paradigm  for  the  optimization,  design,  and  on-line  control  of  DEDS. 

The  intellectual  foundation  and  visionary  aspect  of  the  subject  of  DEDS  have  been  previously  described  and 
can  be  most  easily  accessed  via  the  videotape  tutorial  (#HV0238-6  by  Y.C.  Ho)  published  by  the  ifff. 
Educational  Services. 

Harold  Kushner 

Professor  Harold  Kushner  and  his  students  wtnked  on  a  rigorous  and  systematic  qtproach  to  the  problem  of 
optimal  tracking  via  "minimizing  escape  probabilities"  over  a  given  time  interval,  for  small  noise  systems, 
using  the  theory  of  large  deviations;  diffusion  qtproximation  for  queues,  production  and  rqxur  systems 
woridng  in  a  heavy  traffic  environment;  and  aiqxoximations  to  optimal  repair  and  maintenance  policies  for 
the  above  based  on  noisy  samples  and  small  infotmation  per  sample. 

Rrofessor  Kushner  and  his  co-woikera  sought  numerical  mnhods  for  a  great  variety  of  stochastic  control 
problems.  Their  method  involves  the  systematic  anxoximaiion  of  the  optimal  control  process  by  a  simpler 
one  which  is  manageable  from  a  computational  point  of  view.  In  particular,  the  use  of  some  Marla>v  chain 
modeb  enables  the  utilization  of  a  wide  literature  on  the  solution  of  Markov  problems.  The  approximation 
must  be  conrisient  with  the  underlying  diffusion  or  jump-diffusion  in  the  sense  that  the  local  properties 
must  match  for  the  controb  of  interest.  Both  the  drift  and  the  variance  terms  can  be  controlled;  thb  b,  in 
fact,  the  first  appearance  of  methods  which  can  be  used  when  the  variance  b  controllaUe.  Modeb  that 
require  such  control  occur  all  through  "financial  economics,"  waveform  design,  the  control  of  observer 
location  in  filtering  problems,  etc.  A  great  deal  of  attention  was  given  to  numerical  methods  for  use  in 
production  and  queueing  systems  under  heavy  traffk.  The  proofs  are  very  natural  from  the  point  of  view  of 
the  probabilistic  problems  that  ate  considered,  and  give  considetaMe  insight  into  the  nature  of  the 
qipioximation  procedure  and  the  sorts  of  results  which  can  be  obtained. 

Petros  Maragos 

During  thb  period  Professor  Nbtagos  and  hb  collaborators  performed  research  in  the  areas  of  rignal 
processing,  computer  vision,  computer  speech,  and  ptutern  recognition.  In  the  «ea  of  signal  processing, 
the  new  developments  include  a  unifying  rqnesentation  theory  for  large  classes  of  nonlinear  filten  using 
mtxphological  systems;  a  theory  and  an  efficient  algcvithm  for  computing  the  fractal  dimension  of  aihittary 
signab  using  multiscale  morphological  filters  (with  F.-K.  Sun);  and  nonlinear  partial  differential  equations 
of  the  evolutionary  type  that  model  the  dynamics  of  multiscale  mor{rfK>logical  filtering  in  scale  space  (with 
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R.  Biockett).  The  work  in  the  area  of  computer  vision  includes  the  ai^lication  of  moiphotogical  size 
distributions  to  multiscale  nonlinear  image  smoothing,  shape  representation  and  description;  an  efficient 
system  for  region  feature  extraction  and  segmentation  in  moving  imagery  with  related  correspondence 
algtmthms  (with  C-S.  Fuh  and  L.  Vincent);  a  robust  image  matching  algtmthm  based  on  an  affine  model 
(with  C.-S.  Fuh);  and  methods  for  analysis  and  modeling  of  fractal  images  using  morphological  systems.  In 
the  area  of  pattern  recognition  several  interesting  relationships  have  been  found  between  machine  learning 
theory  and  mathematical  morphology.  The  min-max  pattern  classifiers  were  developed  as  extensions  of 
Botdean  classifiers  for  real-valued  features,  and  apidied  to  the  recognition  of  hand-written  character  images; 
their  leamability  properties  were  studied  (widi  P.-F.  Yang). 

The  work  in  computer  speech  processing  and  recognition  includes  the  use  of  fractals  for  feature  extraction 
and  modeling  -  e.g.  the  use  of  the  short-time  fractal  dimensions  of  speech  signals  for  segmentation, 
interpolation,  and  recognidon  (with  M.  Lu),  and  the  use  of  firactal  signals  as  excitation  of  stochastic  speech 
coders  (with  K.  Young).  In  addition,  a  new  algorithm  has  been  found  for  tracking  amitiitude/frequency 
modulations  in  speech  resonances  based  on  energy  operators  (with  J.  Kaiser  and  T.  Quatieri).  The  low 
computational  complexity  and  efficiency  of  this  algorithm  for  amplitude  and  frequency  detttodulation,  as 
well  as  its  instantaneously-adapting  nature,  make  it  very  useful  for  tracking  modulation  patterns  in  time- 
varying  signals. 

Sanjoy  Mitter 

Professor  Sanjoy  Mitter's  main  research  has  been  (mi  the  study  of  percqttual  systems  (image  analysis  and 
vision,  speaker  independent  speech  recognition,  off-line  handwritten  character  recognition)  and  on  cognition 
(learning).  In  image  analysis  and  vision  he  has  worked  on  boundary  detection  using  variational  and 
probabilistic  methods,  texture  segmentation  using  multi-scale  probabilistic  ideas  and  most  recently  on 
developing  a  language  for  low  and  mid-level  vision.  In  speech  recognition  he  is  cooperating  with  Ken 
Stevens  of  MJ.T.  in  developing  a  qreech  recognition  methodology  which  uses  a  priori  knowledge  in  the 
form  of  linguistic  features  and  incorporates  it  in  an  iqrproptiate  probabilistic  framework.  Hiswotkon 
handwritten  character  recognition  uses  an  informatkm-theor^  framework  for  the  recognition  of  handwrioen 
character  recognition. 

In  the  area  of  learning  his  main  work  has  been  in  tire  extension  of  the  PAC  leanting  framework  of  Vapnik 
and  Valiant  to  solve  problems  in  stochastic  geometry  using  learning  ideas. 

In  a  series  of  pqrers  written  jointly  with  Saul  Gelfroid  (Purdue  University,  fotmeriy  at  QCS)  he  has 
studied  stodiastic  recursive  algtvithrru  (stochastic  gradient-like.  Metropolis)  for  the  minimization  of  a 
function  in  R^. 

A  number  of  research  assistants,  center  fellows  and  post-doctoral  associates  have  contribuied  to  this 
research.  These  include  SanjeevKuOcanii,  Pietro  Perona,  Thomas  Ridrerdson,MohamedAkra,Sie&no 
Cssadei  and  EQen  Eide. 

David  Mnmford 

Professor  Mumford  and  his  students  (Yang  Wang,  Gaile  Gordon,  Mark  Nitzberg,  Peter  Hallinan,  Peter 
Belhnmeur,  Tai-sing  Lee,  and  David  Fry)  have  been  working  the  area  of  computer  vision  atxl  animal/humm 
vision. 


A  stfestantial  part  of  this  worit  has  concerned  development  of  a  Bayesian  approach  to  figute^gtound 
separation,  due  to  S.  and  D.  Geman,  A.  Blake,  A.  Zisserman,  and  D.  Mumfoid.  This  approach  uses  ideas 


18 


24635-MA-6»/y^ 


from  statistical  physics  and  works  by  constructing  a  Gibbs  probability  distribution  on  the  space  of  all 
images  and  their  natural  segmentations.  Prof.  Mumford  and  Jayaiu  Shah  investigated  a  continuous  analog  of 
the  lattice  based  random  fields  of  statistical  physics.  Yang  Wang's  work  investigated  limits  of  this 
algorithm  which  are  close  to  the  classical  "split  and  merge”  segmentation  algorithms  and  also  found  key 
estimates  for  the  problem  of  proving  regularity  of  the  maximum  liklihood  segmwentations.  Made 
Nitzberg's  work  extended  the  above  model  to  analyze  occlusions  in  segmented  images,  using  "T-junctions" 
(points  where  one  object  edge  disappears  behind  a  nearer  object)  as  the  major  clue.  Peter  Beihumeur's  work 
extends  these  ideas  to  stereo  data,  using  a  Bayesian  prior  and  "unnuuched  pixels"  (points  visible  to  one  eye 
only)  as  the  major  clue.  Tai-sing  Lee's  woik  extends  these  ideas  to  texture  segmentation,  using  Gabor 
filters,  with  special  emphasis  on  FM  signal  modulation:  the  presence  of  slowly  varying  spatial  frequencies 
resulting  from  textures  present  on  sloping  surfaces. 

A  second  line  of  research  has  pursued  the  problem  of  recognition  of  shape.  There  are  many  questions  here: 
What  is  the  meaning  of  the  phrase  "two  shapes  ate  similar"?  What  "features”  of  a  shiq)e  are  most  useful  for 
recognition?  How  does  one  match  one's  models  of  shapes  with  the  data?  We  have  used  especially  location 
and  recognition  of  faces  as  a  good  instance  of  these  problems.  Gaile  Gordon  worked  on  recognition  of 
individuals  from  laser  lange  data  of  their  faces.  She  analyzed  the  use  of  the  principal  curvatures  and  other 
differential-geometric  invariants  for  recognition.  Peter  Hallinan,  on  the  other  hand,  has  been  working  on  the 
problems  of  finding  the  optimal  match  between  an  intensity  template  for  a  face,  in  which  various  degrees  of 
freedom  are  allowed,  with  an  unknown  image.  Finally  David  Fry  has  taken  leaves  rather  than  faces  as  a  test 
bed,  and  has  developed  an  algorithm  of  species  recognition  based  on  matching  which  allows  reshuffling  of 
parts,  but  penalizes  the  amount  of  translation  and  rotation  of  each  part  before  reassembling. 

A  final  line  of  research  concerns  analogies  between  the  problems  and  algorithms  of  computer  vision  and  the 
data  from  animal  and  human  vision:  psychophysical,  neurophysiological  and  neuroanatomical.  Prof. 
Mumford  has  developed  a  theory  of  what  algorithms  are  embodied  in  what  parts  of  the  btain  which  is  based 
on  many  known  experimental  results  and  which  inedicts  others. 

Ronald  L.  Rivest 

Professor  Ronald  L.  Rivest  worked  on  the  theoretical  foundations  of  machine  learning.  He  studied  a  number 
of  different  models,  ranging  from  neural  nets  to  distribution-free  learning  paradigms  to  the  inference  of 
finite-state  systems. 

With  Javed  Aslam,  he  worked  on  the  algorithmic  inference  of  Maikov  chains.  With  Sally  Goldman  and 
Robert  Schqtiie,  he  studied  the  problem  of  learning  a  binary  relation,  i.e.  a  predicate  relating  the  elements 
of  two  sets.  With  Robert  Ramstad,  he  worked  on  an  efficient  leimplementation  of  Dieseber’s  learning 
method,  which  provides  a  computational  interpretation  of  Piaget's  theory  of  early  human  lewning.  With 
Yigun  Yin,  Rrofessor  Rivest  developed  and  experimentally  tested  a  new  heuristic  for  the  "two-armed  bandit 
problem.*  With  Tonu  Kuh  and  Thomas  Petsche,  he  studied  the  problem  of  learning  concepts  that  "drift"  or 
"vary*  slowly  over  time.  With  Bonnie  Eisenberg,  he  worked  on  learning  from  both  random  examples  and 
membership  queries,  and  proved  that  membership  queries  are  not  generally  usefuL  unless  there  is  prior 
knowledge  about  the  probability  distribution  according  to  which  the  random  examples  are  drawn.  With 
Robert  Schapire  and  Michael  Kearns,  he  studied  some  problems  related  to  Valiant's  distribution-free  learning 
modeL  and  discovered  some  new  approaches  to  learning. 
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Charles  Rockland 

Charies  Rockland  and  his  coUaboiatocs  condnued  woik  on  the  partial  models  and  constraints  framework  frnr 
rqxesenting  complex  system  organization.  A  basic  emphasis  this  research  was  to  clarify  the  differences 
in  the  formalisms  required  for  natural  vs.  artificial  systons.  This  work  took  two  forms: 

0)  They  continued  their  exploration  of  analogies  with  the  so<alled  principles  and  parameters  approach 

to  natural  language. 

(u)  In  collabotation  with  Bernard  Gaveau  (University  of  Paris,  VI),  they  began  investigation  of  the 
issues  underlying  the  behavioral  qxcification  of  complex  interactive  systems.  Part  of  this  work 
centered  on  identifying  the  circumventing  deficiences  of  existing  formalisms,  such  as  Milner's 
CCS,  develqied  for  artificial  systems. 

John  Tsitsiklis 

Professtx' John  Tsitsiklis  and  his  co-workers  have  carried  out  research  in  a  number  of  areas  disriKiMvi  below, 
as  well  as  certain  operation  research  apidications  and  distributed  detection  and  estimation. 

In  the  area  of  parallel  and  distributed  computation,  they  have  carried  out  a  comprehensive  study  of 
asynchronous  distributed  algorithms  for  optimization  and  other  numerical  problems.  They  have  developed  a 
comprehensive  convergence  theory,  and  have  also  studied  issues  related  to  qwed  of  convergence.  Amain 
conclusion  of  this  study  is  that  a  piethtaa  of  useful  algorithms  retain  their  desirable  convergence  properties 
in  the  presence  of  asynchronism  and  fairly  chaodc  interprocessor  communication. 

On  another  tcqiic,  they  have  formulated  and  solved  a  number  of  generic  routing  and  scheduling  proUena 
that  arise  when  we  try  to  optimize  interiRocessor  communication  in  parallel  architectures,  espedaUy 
hypercubes. 

A  great  part  of  the  work  in  these  areas  has  been  incorporated  in  a  major  textbook  on  parallel  and  distributed 
computation  co-authored  by  Professors  Bertsekas  and  Tsitsiklis. 

In  the  area  of  simulaied  annealing.  Professor  Tsitsiklis  and  his  coUaborators  have  provided  an  altenuuive 
convergence  proof  of  convergence.  By  establishing  a  liidt  with  the  theory  of  Markov  chains  with  rare 
transitions,  this  proof  has  provided  a  new  way  of  understanding  the  functko  of  diis  algorithm. 

In  the  area  of  control  theory  and  computational  complexity,  a  number  of  problems  in  contttd  theory,  such 
as  supervisory  control  of  discrete-event  dynamical  systems  and  discrete-time  stochastic  control,  have  been 
stntlied  from  the  viewpoint  of  computatkxud  complexity  theory.  For  a  concrete  example,  they  have  shown 
that  a  one-way  multigrid  algorithm  is  optimal,  in  a  precise  sense,  for  solving  discrete-time  stochastic 
control  problems  for  the  case  of  continoous  state  qtaces. 

Leslie  Valiant 

Rofessor  Leslie  Valiant  investigated  the  question  as  to  which  classes  of  concept  representation  it  is 
computationally  feasible  to  learn  from  examples.  Leamabillty  is  not  monotonic  in  the  class  of 
representations;  if  the  latter  is  enlarged,  there  is  mote  to  be  learned,  but  the  task  may  become  easier  because 
there  is  now  a  richer  representation  for  expressing  it  fri  collaboration  with  oo-woikers.  Professor  Valiant 
obtitfned  some  general  properties  of  leamable  classes.  In  particular,  they  obey  certain  closer  properties  and 
ate  preserved  under  certain  substitutions.  Also,  strong  statements  can  be  made  about  the  effea  of  having 
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both  positive  and  negative  examples  available,  as  opposed  to  examples  (rf  just  one  kind.  The  effect  in  some 
instances  is  to  render  leamable  what  otherwise  is  not  In  other  cases,  it  reduces  the  number  of  examples 
required,  or  it  increases  the  error  rate  diat  can  be  tolerated  in  the  data. 

Alan  Willsky 

During  the  period  covered  by  this  grant.  Professor  Willsky,  Dr.  Karl,  and  their  students  engaged  in  research 
in  discrete-event  systems  and  in  a  number  of  problem  areas  in  multidimensional  statistical  signal  processing 
and  estimation.  In  the  area  of  discrete-event  systems,  the  major  accomplishtnenu  ate  summarized  in  the 
Ph  J>.  thesis  of  Dr.  Cuneyt  Ozveren  and  in  a  series  of  p^rs  published  by  Prof.  Willsky  and  Dr.  Ozveten. 
This  research  developed  a  notion  of  stability  for  discrete-event  systems  and  explored  its  consequences  in 
feedback  system  design,  system  monitoring,  compensator  design  for  command  frdlowing.  and  hierarchical 
modeling  based  on  the  concept  of  macro-events  or  "tasks*.  Both  Dr.  Ozveren  and  Prof.  Willsky  have  given 
several  invited  talks  on  this  woric  including  Dr.  Willsky's  {veseniation  at  the  1991  ARO-Concepts  Analysis 
Agency  Workshop  on  Combat  Nfodeling.  Dr.  Ozveten  also  received  honorable  mention  in  the  IEEE 
Control  Systems  Society  Student  Paper  Cbntest  for  a  conference  paper  based  on  the  initial  results  in  his 
thesis. 

The  research  of  Prof.  Willsky  and  Dr.  Karl  in  multidimensk>nal  statistical  signal  processing  and  estimation 
has  several  compcMients.  The  first  of  these  is  in  the  development  of  efficient  and  highly  parallelizable 
algmithms  for  multidimensional  estimation.  In  particular,  essentially  all  multidimmisional  estimation 
problems  involve  potentially  explosive  computational  demands,  caused  both  by  working  in  multiple  qiatial 
dimensiona  and  by  the  compounding  lack  of  an  obvious  notion  of  recursion.  The  woric  in  this  area,  as 
summarized  in  the  Ph.D.  theses  of  Dr.  Darrin  Taylor  and  Dr.  T.  Michael  Chin,  and  the  S.M.  thesis  of  Mr. 
KGchael  Daniel  (in  progress)  as  well  as  in  several  papers  written  by  Prof.  Willsky  together  with  Prof.  B.C. 
Levy  (University  of  Califcnnia  at  Davis)  and  Dr.  R.  Nikoukhah  (INRIA,  France),  deals  both  with  new 
notions  of  recursive  computation  in  multiple  dimensions,  with  parallelization  achieved  by  data  partitioning 
and  a  generalization  of  the  technique  of  nested  dissectirn  fv  solving  partial  differential  equations,  and  with 
effective  methods  for  dealing  with  processes  that  evolve  in  time  as  well  as  in  space.  In  particular  a  new 
approach  using  radial  recursions  has  been  devek^  and  demonstrated  to  yield  efficient  and  accurate 
solutions  which  could  then  be  coqtled  with  spatial  data  partitioning  to  produce  a  new  class  of 
multidimensional  parallel  processing  algorithms.  Intheateaof  space-time  processes,  a  key  contribution 
was  the  recognition  that  the  major  computational  problem,  namely  that  of  calculating  error  covariance 
functions  fof  predicted  spatial  fields  and  using  this  to  qtecify  the  procedure  for  incorporating  new 
measurements,  could  be  viewed  as  one  of  statistical  modeling  of  random  fields.  This  led  to  very  effective 
methods  for  reduced-order  modeling  allowing  the  solution  of  qtace-time  estimation  proUems  of  very  high 
dimension  ~  e.g.  S(X),000.  Professor  Willsky's  experimental  research  in  this  area  has  demonstrated  not 
only  the  efficiency  but  also  the  accuracy  of  the  solutions  produced  by  this  method  for  the  problem  of 
motion  estimation  in  image  sequences.  In  addition,  his  wwk  on  multiresolution  methods  described 
subsequently  offers  even  greater  computational  efficiencies,  which,  when  coupled  with  the  research  just 
described  now  makes  possible  the  consideration  of  problems  of  sizes  for  larger  than  any  considered 
previously. 

A  second  component  of  his  research  in  multidimensitmal  processing  and  estimation  is  in  the  foundations, 
theory,  and  application  of  a  methodology  for  multiresolution  statistical  modeling  and  estimation  motivated 
by  and  closely  related  to  wavelet  transforms.  This  work  is  documented  in  the  Ph.D.  theses  of  Dr.  Kenneth 
Chon,  Eric  Miller,  Mickey  Bhatia,  and  Mark  Luettgen  (in  progress)  and  the  M.  Eng.  theses  of  Stuart 
QtAlen  and  Ilya  Polyak  (in  progress)  as  well  as  in  numerous  papers  by  these  individuals,  Prctf.  Willsky,  and 
Dr.  Karl.  The  basis  of  this  woric  is  the  introductimi  of  a  class  of  scale-recursive  models  for  stochastic 
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processes  and  random  fields.  These  models  lead  to  extremely  efficient  algorithms  both  in  one  and  several 
dimensions.  Indeed  this  work  on  multiiesolution  motion  estimation  has  resulted  in  algorithms  that  have 
constant  complexity  per  pixel,  providing  between  one  and  two  orders  of  magnitude  of  computational  speed¬ 
up  over  previously  developed  methods.  In  addition,  this  multiresolution  modeling  fitamework  is  rich 
enough  to  capture  an  extremely  broad  class  of  processes  and  random  fields.  In  particular,  it  is  possible  to 
model  fractal-lilce  or  so-called  "i//'  processes  in  this  frameworit,  leading  to  a  new  class  c£  "fractal 
regularization"  algorithms.  In  addition.  Professor  Willsky  and  his  collaborators  have  shown  that  a 
generalizatian  of  Levy's  so-called  midpoint  deflection  construction  of  Brownian  motion  provides  the  basis 
for  the  exaa  multiresolution  modeling  of  arbitrary  Mrukov  random  fields  and  for  extremely  compact  and 
computadmially  attractive  apiaoximate  modeling,  opening  up  the  possibility  of  exuemely  efficient 
algorithms  for  task  such  as  image  segmentation  and  texture  identification.  In  addition,  their  multiresolution 
methods  also  offer  considerable  promise  in  the  area  of  inverse  problems,  described  below. 

The  third  component  of  Professor  Willricy's  research  is  in  the  estimation  and  reconstruction  of  geometric 
features  in  multidimensional  data.  This  wmk  is  documented  in  the  S.M.  theses  of  Seema  Jaggi  and  Lori 
Belcastro  (in  progress)  and  the  Ph.D.  theses  of  Dr.  Karl  and  Peyman  Milanfar  (in  progress)  Major 
contribution  in  this  work  include  the  development  of  a  methodology  for  the  reconstruction  of  3-D  objects 
from  their  2-D  silhouettes,  and  the  tracking  of  objects  with  time-varying  shape.  This  methodology  has 
been  proven  to  woric  in  practice  resulting  in  a  new  approach  to  temporal  imaging  of  the  heart  give  very 
low-dose  (and  thus  tow  SNR)  imagery.  They  have  also  developed  new  approaches  to  the  direct  extraction 
geometric  information  from  tomographic  data,  which  also  provide  new  algorithms  in  computational 
geometry  that  direcdy  accommodate,  and  hence  ate  robust  to,  uncertainties  and  errors  in  the  observed  data. 

The  final  part  of  this  research  is  in  the  development  of  statistical  methods  for  inverse  problems  in  one  and 
several  dimensions.  In  particular,  multiresolution  methods  have  been  apidied  to  problems  of  one- 
dimensional  multisensor  deconvolution  and  fusion  (Eric  Miller)  and  tomography  (Mickey  Bhatia),  while  our 
worit  on  geometric  recanstniction  from  tomographic  data  (Peyman  Milanfer  and  Lori  Belcastro)  provides 
new  algorithms  for  inverse  Radon  transforms  based  on  nonlinear  (Le.  geometric)  parametrizations  of  the 
field  to  be  estimated. 

This  group's  work  in  multidimensional  statistical  signal  processing  and  estimation  has  received  considerable 
national  and  international  attention  as  evidenced  by  numerous  invited  lectures  and  keynote  addresses  that 
Prof.  Willsky  has  given  in  this  area  as  well  as  by  his  guest  editorship  of  the  recent  qtecial  issue  of  the 
IEEE  Transactions  on  Iirformation  Theory  on  wavelet  transforms  and  multiresolution  signal  analysis. 
Among  Prof.  Willsk/s  invited  lectures  are  plenary  and  keynote  lectures  at  the  Seventh  Army  Conference  on 
Apftiied  Mathematics  and  Computing  in  1^,  the  1991  IEEE  International  Conference  on  Systems 
Engineering,  the  1991  SIAM  Conferenoe  on  Linear  Algdna  and  Its  Applkatioos  and  the  Opeiting 
Ceremonies  and  Workshop  for  the  National  Coqrerative  Research  Centre  on  Robust  and  Adrqrtive  Systems 
in  Canberra,  Australia  in  February  1992. 

AUn  Yullle 

During  this  period  Professor  Yuille  and  his  collaborators  and  students  performed  theoretical  and 
e^terimental  wtuk  in  the  areas  of  computer  vision  and  neural  networks.  Highlights  of  this  wmk  include: 

(9  the  development  and  the  application  of  deformable  template  models  to  the  detection  of  facial 

features,  automatic  diagnosis  from  medical  images,  and  the  detection  of  particles  in  high  energy 
lAysics  experiments; 
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(ii)  the  Implication  of  Bayesian  ideas  to  computer  vision  including  image  segementmentation  and 
binocular  stereo  vision; 

(iii)  demonstrating  a  relation  between  Bayesian  theories  of  stereo  vision  and  psychophysical 
experiments; 

(iv)  the  development  of  a  Bayesian  framework  for  sensor  fusion  (with  Prof.  J  J.  Clark); 

(v)  self>oiganizing  neural  network  models  for  develcmment  of  structures  in  the  early  visual  cortex;  and 

(vi)  showing  that  by  putting  neural  network  modd  within  a  statistical  physics  framework  one  could 
obtain  mote  effective  algorithms  and  demonstrate  hitherto  unexpected  relationships  between 
existing  models. 

10.0  INTERACTIONS  WITH  ARMY  RESEARCH  OFFICE  AND  ARMY 
LABORATORIES 

Numerous  individual  interactions  between  Army  Laboratory  personnel  and  Center  faculty  take  place  each 
year.  These  interactmns  can  be  categorized  as: 

(i)  S  ubstantive  interactions  extending  over  tong  periods  of  time 

(ii)  Shut  courses  presented  at  the  request  of  Army  Laboratories 

(iii)  Participation  in  Army-uganized  confoences  and  workshops 
The  following  sections  describe  these  interactions  in  greater  detail 

10.1  Substantive  joint  work  extending  over  long  periods  of  time 
Rqxesentative  programs  here  are: 

CICS/Martin  Marietta/Human  Engineering  Laboratory  Joint  Work  on  the  Field 
Material  Handling  Robot  (October  1988  to  date) 

Dr.  Pieire  Dt^ont's  experience  as  a  Postdoctoral  Associate  with  the  Center  is  an  example  of  how  interaction 
between  academia,  industry  and  govenunent  can  prove  beneficial  to  alL  In  reqxxise  to  a  request  fu  Center 
assistance  by  Martin  Marietta  Aero  &  Naval  Systems,  Dr.  Dupont  aided  Martin  Marietta's  engineers  in  the 
dynamic  modeling  and  controller  design  of  the  Field  Materiel  Handling  Robot  G’MR),  being  built  under 
contract  fu  the  US  Army  Human  Engineering  Labuatory. 

In  particular.  Dr.  Dupont  develtmed  friction  models  for  the  transmission  elements  of  the  FMR.  These 
models  were  verified  experimentally  by  Martin  Mmietta  and  included  in  both  their  simulation  software  and 
in  the  FMR's  feedforward  cuitrol  law.  In  addition,  general  procedures  fu  simulating  friction,  eqtecially 
toad-dqiendent  friction,  were  developed. 

TheFMRprqjecthasledDr.  Dupont  to  begin  work  on  several  other  friction-belated  ttqncs.  One  of  these 
involves  the  control  of  steady,  tow-velocity  motion.  While  the  FMR  is  primarily  a  gross-motion  machine, 
there  are  many  situations,  such  as  those  involving  dextrous  manipulation,  when  fine-motiui  control  is 
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pafamount  In  these  cases,  the  transient  effects  of  fincdon,  typically  ignored  by  control  theoists,  play  an 
important  role  in  determining  controller  requirements  for  avoiding  stick-slip. 

A  second  topic  concerns  the  existence  and  uniqueness  properties  of  the  forward  dynamic  solution  with 
Mcdon  ~  the  problem  which  must  be  solved  to  simulate  the  motion  of  a  system.  Friction  can  cause  the 
Newtonian  formulation  of  certain  rigid-body  dynamics  problems  to  break  down.  In  these  situations,  a 
problem  can  appear  to  have  no  solution  or,  through  the  use  of  solid  mechanics,  to  have  multiple  solutions. 
Dr.  Dupont  is  investigating  a  new  class  of  problems  which  possess  multiple  valid  solutions  due  to 
firiction. 

CICS/ARDEC  Joint  Work  on  Systems  Dynamics  and  Control  (June  1990  to  date) 

Dragan  Obradovic,  a  postdoctoral  fellow  with  the  Center,  wr^ced  with  Dr.  Nmrnan  Coleman's  Laboratmy  at 
ARDEC  on  a  project  whose  goal  is  to  improve  the  accuracy  of  the  helibome  gunAurret  system  by  using 
state  of  the  art  control  technology.  This  is  to  be  achieved  by  deriving  a  sufficiently  accurate  system 
dynamics  model  that  captures  all  the  important  phenmnena  occurring  during  the  firing  instances. 

In  particular.  Dr.  Olxadovic  has  worked  on  improving  the  existing  models  of  the  guriAurret  dynamics  and 
adjusting  them  to  a  form  suitable  for  the  robust  control  design.  These  models,  derived  by  Integrated 
Systems  Incorporated  and  Lear  Astronics  Corpmation,  subcontractors  on  the  same  project,  are  proven  to  be 
accurate  only  at  a  very  limited  range  of  frequencies.  Consequendy,  they  do  not  capture  the  flexible  modes 
excited  during  the  flring  that  ate  considered  to  be  the  main  source  of  disturbance  in  the  system.  Therefore, 
the  work  of  Dr.  Obradovic  focuses  on  obtaining  a  better  description  of  the  gunAurret  dynamics  during  the 
firing  by  both  analytical  and  experimental  approaches  and  on  characterizing  the  uncertainty  associated  with 
die  model. 

This  project  has  led  Dr.  Obradovic  to  begin  working  on  other  topics  omoeming  system  identification  and 
model  validation  for  robust  control  design.  In  the  case  of  deterministic,  unknown  but  norm  bounded 
disturbances,  system  identification  in  general  does  not  lead  to  a  single  model  but  to  a  set  of  possible 
models.  Dr.  Obradovic  is  investigating  how  the  a  priori  knowled';^  about  the  actual  system  and  choice  of 
the  input  signal  used  in  an  experiment  influence  the  result  of  the  identification. 

10.2  Short  courses  presented  at  the  request  of  Army  Laboratories 

In  addition  to  regularly  scheduled  Center-organized  woritshops.  Center  fiaculty  have  presented  a  number  of 
short  courses  at  the  request  of  various  Army  Laboratories.  Among  them  are: 

®  Two-dqr  short  course  on  Nonparametric  Statistics  presented  by  Professor  Stuart  Geman  at  the  Army 

Engineering  Waterways  Experiment  Station. 

(u)  Two-day  short  course  on  Theory  of  Large  Deviations  presented  by  Professor  Daniel  Stroock  atthe 

University  of  Maryland. 

(iii)  Two-day  short  course  on  Nonparametric  Statistics  and  Applications  to  Image  Analysis  presented  by 
Professor  Donald  McQure  at  the  U.S.  Army  Engineers  Waterways  Experiment  Station. 

Vicksburg,  Mississippi. 

Qv)  Tutorial  on  Modem  Contrd  Systems  Design  using  the  Q-paiametrization  presented  by  Professor 
Munther  Dahkh  at  ARDEC. 
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10.3  Participation  in  Army-organized  conferences  and  workshops 

Center  faculty  are  regularly  asked  to  speak  in  the  Aimual  Army  Conference  on  Applied  Mathematics  and 
Computing.  For  example: 

At  the  Fifth  Army  Conference  at  West  Point ,  1987,  Professor  Sanjoy  Mitter  spoke  on  "Stochastic 
Quantization,”  Professor  Gilbert  Strang  on  "Structural  Optimization  and  Composite  Material  in  Optimal 
Designs,"  and  Professor  David  Mumford  on  "Some  Mathematical  Problems  Arising  from  Computer 
Vision." 

Professor  Alan  Willsky  spoke  on  "Estimation  for  Multidimensional  Random  Fields”  at  the  seventh  Army 
Conference  in  1989. 

Professor  Mitter  was  again  invited  to  speak  at  the  eighth  Army  Conference  in  1990  on  "Variational 
Problems  with  Free  Discontinuities  and  Nonlinear  Diffusions." 

In  addition.  Center  faculty  are  regularly  invited  to  participate  in  otho'  Army-organized  workshops  and 
conferences,  for  example: 

Professor  Stuart  Geman  spoke  on  "Hidden  Markov  Models  for  Image  Analysis"  at  the  ARO-sponsored 
Encounter  of  Computer  Vision  and  Mathematics  Woricshop,  May  21-23, 1990,  at  the  University  of 
Pennsylvania. 

Professor  Roger  Brockett  presented  an  invited  talk  at  the  Ninth  Army  Mathematics  Conference  in  June 
1991. 

10.4  Individual  Interactions 

Numoous  individual  interactions  between  Army  Labortuory  personnel  and  Center  faculty  take  place  each  year. 
For  example,  over  the  past  year,  the  following  interactions  were  reported: 

Professors  Alan  Willsky  and  Y.C.  Ho  participated  in  a  meeting  in  May  1990  at  the  U.S.  Army  Materiel 
Command  in  Alexandria  on  the  use  of  recent  results  on  discrete  event  dynamic  systems. 

ProfesstH’  Y.C.  Hb  also  had  discussions  with  Richard  Chait,  Chief  Scientist,  Army  Material  Command,  on  May 
22, 1990.  He  met  with  scientists  from  the  Army  Material  Systems  Analysis  Activity  at  the  Aberdeen  Proving 
Ground  in  June  1990.  He  also  visited  the  AMC  Headquarters  in  Washington,  D.C.,  and  the  Aberdeen  Proving 
Ground.  He  interacted  with  Aberdeen  posotmel  on  various  logistics  and  tank  ground  war  problons  involving 
simulation,  and  was  invited  to  speak  on  June  S,  1990. 

Dr.  Paul  Tseng  had  discussions  with  Joel  Levy,  Operations  Research  Analyst  at  the  US  Army  Concqrts  Analysis 
Agency,  on  the  potential  of  using  network  flow  methods  to  solve  a  warehouse  allocation  problem. 

Professor  Munther  Dahleh  has  had  extensive  interaction  with  Norman  Coleman,  ARDEC,  and  CICS  postdoctoral 
fdlow  Dragan  Obradovic,  regarding  a  controller  design  for  a  gun  that  will  be  mounted  on  top  of  an  aircraft  This 
control  problems  highlighted  new  and  interesting  directions  for  research,  particularly  in  the  problem  of 
identification  within  uncertainty.  Dr.  Obradovic  presented  some  of  these  ideas  at  the  meeting  in  November 
organized  by  Dr.  Jagdish  Chandra,  ARO. 
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Professor  Donald  McClure  has  continued  to  consult  with  scientific  staff  at  the  Center  for  Night  Vision  and 
Electro-Opdcs  on  image  processing  problems. 

Professor  Basilis  Gidas  met  with  a  number  of  the  Army  personnel  (including  Dr.  Chandra)  at  W  J.  Schaeffer 
Associates  Inc.,  Rosslyn,  Virginia,  and  discussed  possible  applications  of  CICS  woiit  to  the  Army,  and  in 
particular,  to  SDI  problems.  Prof.  Gidas  also  discussed  his  views  on  new  directions  in  multiresolution-multiscale 
phenomena  in  computer  vision  and  dynamical  systems  in  August  1990.  During  the  fall  of  1990,  Prof.  Gidas  had 
contacts  with  Dr.  Jerry  Andersen  of  ARO  to  establish  connections  with  members  of  the  laboratory  and  industry  in 
relation  to  his  work  on  the  analysis  of  Synthetic  Aperture  Radar  data. 

11.0  MECHANISMS  FOR  EXCHANGING  TECHNICAL  INFORMATION 

It  is  our  goal  that  the  Center  for  Intelligent  Control  Systems  (CICS)  should  serve  as  an  educational  and 
research  resource  for  the  Army  as  well  as  function  as  a  national  center  for  fundamental  research  in  Intelligent 
Control  Systems. 

An  important  aspect  of  the  Center  is  its  role  as  a  meeting  ground  for  researchers  from  the  Army 
Laboatories,  the  Center  and  from  outside.  Interactions  involving  Army  Laboratory  personnel  take  place  in 
scholariy  meetings  such  as  research  reviews,  workshops  and  colloquia,  and,  individually,  in  interactions 
between  Center  faculty  and  army  researches  working  on  particular  problems  or  issues.  Information  about 
research  activities  in  the  Center  is  also  disseminated  in  print,  in  the  form  of  CICS  Technical  Reports,  the 
newsletter  The  Intelligent  Control  Letter,  and  other  mailings. 

The  listing  below  summarizes  the  various  mechanisms  for  the  exchange  of  technical  information  with  the 
Army's  researchers. 

0)  Research  Reviews 

(u)  CICS  Workshops 
(iii)  Colloquium  Series 
Gv)  Publications  Program 

(v)  Interactions  with  Army  Researchers  Oong-term,  short-term) 

In  the  following  sections,  we  document  these  mechanians  in  greater  detail. 

11.1  Research  Reviews 

The  yearly  or  biannual  Research  Reviews  provide  an  overview  of  current  CICS  research  and  include 
technical  presentations  by  both  Army  scientists  and  engineers  and  CICS  faculty.  Because  Research  Reviews 
m  scheduled  for  an  attire  day,  with  time  held  for  informal  contact,  they  are  an  excellent  oppntunity  for 
lengthier  interactions  between  Center  faculty  and  Army  researchers.  Listed  below  are  the  8  research  review 
meetings  that  have  taken  place  thus  far. 

March  2, 1987  Research  Review  at  M.I.T. 

December  17, 1987  Research  Review  at  M.I.T. 
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August  30, 1988  Research  Review  at  M.LT. 

November,  1988  Information  Exchange  Meeting  at  Raytheon  Corporation 

March  24-25, 1989  External  Technical  Review  at  MJ.T. 

September  25-26, 1989  Program  Review  at  M.LT. 

Mtuch  15-16, 1990  Program  Review  at  M.LT. 

October  10, 1990  Infonnation  Exchange  Meeting  at  Harvard  University 

11.2  Workshops 

Twice  annually,  subjects  of  interest  are  reviewed  by  CICS  researchers  and  outside  experts  in  the  context  of 
specialized,  high  level  workshops.  Listed  below  are  the  12  workshops  that  have  been  held  thus  fan 

October  22-24, 1986  Workshop  on  Distributed  Algorithms  in  Communication 

and  Computation 

May  27-29, 1987  Workshc^  on  Mathematical  Aspects  of  Data  Network  Performance 

November  21, 1987  Workshop  in  Computer  Vision 

June  14, 1988  Tutorial  W(»kshop  on  Intelligent  Control  Systems 

(at  the  1988  American  Control  Conference) 

November  5, 1988  Workshop  on  Randomness  in  Computatkai 

May  5, 1989  Machine  Learning  Workshop 

May  10, 1989  Woritshop  on  Image  Analysis  and  Vision 

November  12-14, 1989  Five  Tutorials:  Some  Modem  Applications  of  Mathematics 

January  26, 1990  Wc^tshop  on  Speech  Processing  and  Recognition 

May  15, 1990  Workshop  on  Global  Models  fw  Image  Analysis 

February  25, 1991  Workshop  on  VLSI  and  Computer  Vision 

April  1 , 1991  Workshop  on  Analog,  Neural  and  Asynchronous  Computing 

11.3  CICS  Colloquia  and  Seminars 

A  series  of  monthly  coUoquia  was  initiated  in  February  1987.  The  series  provides  an  opportunity  for  CICS 
researchers  to  interact  with  outstanding  scholars  whose  work  is  related  to  ongoing  research  at  the  Center. 
Documented  below  the  29  colloquia  that  have  taken  place  since  the  establishment  of  the  Center.  In 
addition,  some  40  seminars  have  been  held  at  MIT,  Harvard  University  and  Brown  University. 
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Fd)ruary  1987 
Professor  Ulf  Grenander 
Brown  University 

"Can  We  Understand  Shapes  in  Nature?" 

March  1987 
Professor  Achi  Brandt 
Dqnrtment  of  Mathematics 
Weiznuinn  Institute 
Tast  Optimization  and  Fast  Statistics" 

May  1987 

Dr.  Shlomo  Ta’asan 

ICASE,  NASA  Langley  Research  Center 
"A  Multigrid  Method  for  Slightly  Indefinite 
Problems  withApplication  to  the  Riccati  Equation" 

October  1987 

Professor  Karl  Johan  Astrom 

Lund  Institute  of  Technology,  Lund.  Sweden 

Toward  Intelligeru  Ccmtrollers" 

December  1987 

ProfesstK'  Stephen  Grossberg 

Center  for  Adaptive  Systems,  Boston  University 

"Emergent  Invariants  of  Neural  Networics  for 

Adquive  Pattem  Recognition" 

December  1987 
Professor  Robert  Berwick 
Artificial  Intelligence  Laboratory,  M.LT. 
"Computation  and  Language  Acquisition" 

March  1988 

Professor  Persi  Diaconis 
Mathematics  Dqrartment,  Harvard  University 
"Projection  Pursuit:  An  Approach  to  Visualizing 
in  High  Dimensions" 

Ai)rU1988 

Professor  Brian  D.  Ripley 
Department  of  Mathematics 
University  of  Strathclyde,  Glasgow 
"Mathematical  Morphology  in  Image  Analysis" 

May  1988 

Professor  Silvio  Micali 
M.LT. 

"hitetactive  IftooCi,  Zero-Knowledge 
andApfriications" 


October  1988 
George  N.  Reeke,  Jr. 

The  Neurosciences  Institute  and 
the  Rockefeller  University 
"Selective  Recognition  Automata" 

October  1988 
Dr.  M  JC.  Sparrow 
Harvard  University 

Topological  Coding  of  Single  Hngerprints" 
October  1988 

Professor  Ji.  Wyatt,  Jr.,  M.LT. 

"Circuit  Design  Criteria  for  Stability  in  a  Class 
of  Lateral  Inhibition  Neural  Networks" 

November  1988 

Professor  Roger  W.  Brockett,  CICS 
"Differential  Equations  that  Sort  and 
Solve  Linear  Programming  Problems" 

November  1988 
Dr.  Nfichael  Barnsley 
Iterated  Systems  Inc.  and 
"Gewgia  Institute  of  Technology 
Iterated  Function  Systems" 

January  1989 
Dr.  JJL.  Lions 

College  de  France  and  Centre  Natiotud 
des  Etudes  Spatiales 

"Special  Functionals  for  Distributed  Systems" 
March  1989 

Professev  R.T.  Rockefellar 
University  of  Washington,  Seattle 
"Extended  Linear-Quadratic  Programming 
and  Large-Scale  Optimization" 

March  1989 
Dr.  Ingrid  Daubechies 
ATftT  Bell  Labonttories 
"Wavelets:  New  and  Old  Results" 

Match  1989 
Professor  John  S.  Baras 
Systems  Research  Center 
University  of  Maryland 
"Architectures  for  Real-Time  Sequential 
Detection  and  Estimation" 
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1989 

Pitofessw  Eugene  Wcmg 
University  of  Califonia,  Bericeley 
"Stochastic  Neural  Networks" 

October  1989 
Nicolas  Sourlas 

Ecole  Normale  Supetieure,  Paris 
"Spin-Glass  Models  and 
Error-Conecting  Gxles" 

November  1989 

Bernard  Gaveau 

University  of  Paris  VI 

"Rate  Constants  for  Meta-stable  Systems" 

December  1989 
Stqthane  Mallat 
New  York  University 
"Complete  Signal  Rqnesentation 
with  Multiscale  Edges" 

February  1990 
FtederickJeltnek 

IBM  TJ.  Watson  Research  Center 
"Language  Translation  by  Statistical  Methods" 

March  1990 
NfichadShub 

IBM  TJ.  Watson  Research  Center 
"P  and  NP  in  Various  Contexts: 

Hubert's  NuUsteUensats  and 
Diophantine  Equations" 

11.4  Honors,  Awards  and  Invited  Lectures 
CICS  Faculty  at  Brown  University 
WeodeU  Fleming 


October  1990 
YX  Ysyirfdn 

Acacemy  of  Sciences,  Moscow 
"Control  of  Systems  with  Uncertainty: 

Past,  Present  and  Future" 

October  1991 
R.E.  Kalman 

Swiss  Federal  Institute  of  Tech.,  Zurich 
"A  New  Lode  at  Randomness  as  a 
System  Phenomena" 

November  1991 
Shankar  Sastry 

University  of  CalifcHiiia,  Berkeley 
"The  Art  (Control  Theory)  of  Parking" 

November  1991 
Thomas  Kailath 

Suuifotd  University,  Stanford,  CA 
"Aiq>lications  d  Signal  Processing  and 
Control  to  Semiconductor  Processing" 

December  1991 
O.  David  Forney 

Motorola  based  at  Codex  Corporation 
"Dynamics  of  Linear  Systmns  over  Gamps" 


-  Steele  Mae  of  the  American  Mathematical  Society,  1987. 

•  A  special  issue  of  the  SIAM  Journal  on  Control  and  Opdmxadon 
will  be  dedicated  to  Wendell  Fleming. 


CICS  Faculty  at  Harvard  University 

Roger  Brackett  -  1989  Richard  E.  Bellman  Control  Hoitage  Award  of  the  American  Control 

CouncU,  1989. 

•  ElectedtotheNadonal  Academy  of  Engineering,  1991. 

Janies  J.  Clark  -  1987  IEEE  Acoustics,  Speech,  and  Signal  Processing  Society  Paper  Award. 
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Y.C.  Ho  •  1989  IEEE  I^kl  Award  for  Coolrol  Science  and  Engineering. 

-  Appointment  to  the  Visiting  Cockrell  Family  Regental  Chair  in  Engineering 
at  the  University  of  Texas  at  Austin.  1988. 

•  Appointment  to  the  T.  Jefferson  Coolidge  Professorriiq)  of  Applied 
Mathematics  at  Harvard  University.  1988. 

-  ElectedtotheNadooal  Academy  of  Engineering.  1987. 

Petros  Matagos  -  IEEE  Acoustics.  Speech  and  Signal  Processing  Society's  Paper  Award.  1988. 

•  National  Science  Foundation  Presidential  Young  Investigator  Award.  1987. 

David  Mumfoni  •  MacAithur  FeUowship.  1987. 

Leslie  Valiant  •  Nevanlina  Prize.  1986. 

CICS  Faculty  at  M.I.T. 

Munther  Dahleh  -  George  Axelby  best  paper  award  of  the  IEEE.  ^)ril  1987. 

-  National  Science  Foundation  Principal  Young  Investigator  Award.  1991. 

Robert  Gallager  •  IEEE  Medal  of  Honor.  1989. 

-  Appointment  to  the  Fujitsu  Professorshq)  of  Electrical  Engineering  and 
Computer  Science.  M.I.T.,  1988. 

ShafiGoldwasaer  •  National  Science  Foundadon  Presidential  Young  Investigator  Award.  1987. 

Sanjoyhfitter  •  ElectedtotheNadonal  Academy  of  Engineeting.  1988. 

RonaldRivest  •  ElectedtotheNadonal  Academy  of  Engineering.  1990. 

Gunicr  Stein  •  1989  Bode  Prize  of  the  Control  Systems  Society. 

John  Tsitsiklis  •  MITsEdgetton  Award  for  excepdonaldisdncdon  in  leaching,  research  and 

scholarship.  1989. 

-  National  Science  Foundation  Principal  Young  Investigator  Award.  1986. 

•  Control  Systems  Society  Best  Piqwr  Award,  1986. 

AlanWiOsky  •  Distinguished  Mdnber  Award  by  the  IEEE  Control  Systems  Society.  1988. 

CcBtcr  Fellows  and  Research  Assistants 

BonniBBeqBer  -  Matchey  Award  for  the  best  studempqrer,  30th  IEEE  Annual  Symposium  on 

the  Foundations  of  Compotff  Science.  1990. 

•  2-year  NSF  Mathematical  Sciences  Postdoctoral  Research  FeUowsh^ 

ChneytOzveren  -  Rnalist  in  the  1989  IEEE  CDC  Best  Paper  Award  Student  Compeddon. 
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